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1 introduction : MIL-D-1000 Drawings, Engineering & Associated Lists. This Military 

Specification prescribes general requirements for the preparation of Engineering 
Drawings 3c Associated Lists, and for application of Intended Use Categories for 
L their acquisition. The key word to be considered when analysing the impact of 
‘HIL-D-1QOO is the word general . The specification does indeed set forth its provis- 
ions in general terms, but in keeping with current Military Specification practices 
i cites as requirements a whole hierarchy of other specifications and standards. These 
[supplementary documents particularize all of the details which are to be included in 
jthe preparation, reproduction, and delivery of drawings the US Government contracts 
to purchase. DISC has studied the cost driving aspects of this Military specificat- 
ion in the previous Low Cost Program Practices study, but the study was cursory only, 
land the cost factors derived- were based upon limited data. This analysis of MIL-D-1000 
investigates the complex interactions of the entire documentation hierarchy to a much 
greater level of detail, and furnishes not only cost values based upon actual program 
experience, but also recommends changes to the extant MIL-D-1000 specification believed 
sssential for improvement of the utility of the document as a means of drawing control. 
historical Background ; MIL-D-1000 superseded a previous document MIL-D-70327 as of 
jlarch 16 1959. MIL-D-70327 had been in use for almost 20 years at that time, hut 
jiuch confusion existed among contractors concerning its application, as it applied 
Essentially to drawings from which small components were made and assembled. No 
particular standardization of symbology was involved, nor was the requirement for 
[oproducibility of drawings clearly defined. MIL-D-1000 was an attempt upon the part 
|f Military procurement agencies to rectify the faults, and ooviate the shortcomings 
p£ its predecessor. Unfortunately, the task of preparation was entrusted to Army 
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FOREWORD 


This Volume II is the Appendix to Volume I, Low Cost Program Practices 
Final Report. As such, it contains those Engineering Memoranda generated 
during the course of the 18 month LCP? study contract NAS W-2752. EMs 
#7 and #18 were not included in this Volume by direction from NASA/HQ 
Low Cost Systems Office (LCSO), hence the breaks in the chronological sequ- 
ence. All others EMs included follow in ascending numerical order in accordance 
with the Table of Contents which appears as Page (ii) of this Appendix. 
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Introduction : MIL-D-1000 Drawings, Engineering Si Associated Lists. This Military 
Specification prescribes general requirements for the preparation of Engineering 
Drawings Sc Associated Lists, and for application of Intended Use Categories for 
their acquisition. The key word to be considered when analysing the impact of 
MIL-D-1000 is the word general . The specification does indeed set forth its provis- 
ions in general terms, but in keeping with current Military Specification practices 
cites as requirements a whole hierarchy of other specifications and standards. These 
supplementary documents particularize all of the details which are to be included in 
the preparation, reproduction, and delivery of drawings the US Government contracts 
to purchase. LMSO has studied the cost driving aspects of this Military specificat- 
ion in the previous Low Cost Program Practices study, but the study was cursory only, 
and the cost factors derived- were based upon limited data. This analysis of MIL-D-1000 
investigates the complex interactions of the entire documentation hierarchy to a much 
greater level of detail, and furnishes not only cost values based upon actual program 
experience, but also recommends changes to the extant MIL-D-1000 specification believed 
essential for improvement of the utility of the document as a means of drawing control. 
Historical Background : MIL-D-1000 superseded a previous document MIL-D-70327 as of 

March 16 1 9^9 • MIL-D-70327 had been in use for almost 20 years at that time, but 
much confusion existed among contractors concerning its application, as it applied 
essentially to drawings from which small components were made and assembled. No 
particular standardization of symbology was involved, nor was the requirement for 
reproducibility of drawings clearly defined. MIL-D-1000 was an attempt upon the part 
of Military procurement agencies to rectify the faults, and obviate the shortcomings 
of its predecessor. Unfortunately, the task of preparation was entrusted to Army 
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Maintenance Unit cognizance. These agencies were concerned primarily with 
drawings intended for Maintenance and Logistics operations and support, as 
pertaining to aircraft, army ground vehicles, and weaponry. The pec uli ar re- 
quirements of space vehicles were not foreseen at the time of MIL-D-1000 
preparation and issue, and even at the time of its revision, March 1 1965, 
no account was taken of the complexities of requirements built into the 
specification, and their cost impact upon space programs. The primary intended 
use of the specification was to faciliate procurement by the US Government 
of Master drawings for production of as many as l£0 legible copies of each 
drawing involved. Further, these Master drawings would be so standardized 
that all government contractors would be able to produce hardware from a 
drawing, or set of drawings, regardless of where these had been produced, or by 
whom. This primary intention has not been very successful. 

IilL-D -1000, ^oimis^^d^Gategories : Fig.l. MIL-D-1000 Associated Specifications, 

Standards, Handbooks & Manuals Tree, depicts graphically, the interdependency 
and inter-relationship of the many documents comprising the hierarchy invoked 
by MIL-D-1000. Problems arise when contractors attempt to interpret this specif- 
ication since there are three Forms implicit in MIL-D-1000, any one of which 
can be invoked by the procuring agency. Form 1. . the most rigorous, imposes 
the entire hierarchy. Form 2 .. curtails approximately $0% of the requirements 
imposed by Form 1,, but leaves in force those provisions and requirements 
causative of the greatest costs to a program; whereas Form 3 negates the require 
ments of MIL-rD-1000 almost entirely, retaining in .force only the safeguarding of 
classified information rqmt, the identification of rights to data provision, 
and the inspection to assure compliance with instructions clause. Fig. 2. lists 
requirements by category to permit easy comparison. 
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MIL-D-1000 Cost Driving Provisions ; There are two major cost drivers implicit 
within this specification: 

(1) The imposition of MIL-STD-100A Engineering Drawing Practice . This 
standard is imposed in full when MIL-D-10Q0 Form 1, is mandated, 
and is imposed in part, to the degree stated in MIL-D-1000, when 
Form 2 is specified. MIL-STD-100A is not invoked by Form 3- 

(2) MIL-Q-9858A Quality Program Requirements. MIL-D-1000 imposes this 
specification for all three forms of drawing control, and all draw- 
ing categories included in the specification. 

Of these two, the second, MIL-Q-98£8 a is the greater cost driver for most programs 
and evidence will follow which indicates that the costs to assure that require- 
ments have been met, almost equal in amount, costs to prepare the drawings required 
for the program, under consideration. 

Cost Comparisons Based on Program Data: As has 'been stated, MIL-D-1000 was 

intended primarily for use with aircraft drawings where large numbers of 
reproducibles were to be produced to a given standard, so that large numbers 
of copies could be obtained per master, and that several different contractors 
from the one producing the drawings could fabricate hardware from a set of these 
standard drawings furnished by the Government. Space vehicles often are "one of 
a kind", and employ end item hardware of a type not envisioned when the Military 
specification was compiled. Symbology for example, did not exist for items which 
have been developed for space usage, when MIL-D-1000 was drawn upj and drawing 
types for printed circuit masters were not included in the classes of drawings 
governed by application of MIL- ST D-100A. The outcome of these deficiencies in 
the specif cation and standard was that for most space vehicle programs under- 
taken by aerospace contractors. Form 1 was never imposed. Firm 2 was the usual 
method of control as it permits more latitude in the preparation of drawings, 
and imparts considerably less penalty to tight schedules. Costs of compliance 
with the . requirements of Form 1 ace thus almost impossible to acquire. 
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LMSC has however, undertaken programs requiring Form 1 type of control. One 
of these was the recently completed Y0-3A Quiet Aircraft contract for the US 
Army. While this is an aircraft, the on-board systems are built to space 
standards for high reliability parts, long maintenance-free life, and 
survivability. LMSC considers that this contract shows the trend of costs 
when full compliance with Form 1 requirements is mandated by a Government 
contracting agency. For details of these costs see the cost breakdown given in 
Figure 3. The significant cost driver is evident immediately: 

o The cost of quality inspection of the engineering drawings produced 
was hk*9 of the total drawing costs, which in turn accounted for 
10*9 of the total program costs. (Development program only, not prod- 
uction follow-on contract.) 

. o The program still carries a classification of Secret, so many details 
cannot be considered in this E.M. However, a good comparison may be 
made in the area of engineering drawings as complete sets of drawings were 
furnished to LMSC by the Airframe manufacturer. This manufacturer, a 
major U.S. supplier of light commercial aircraft provided drawings for 
the aircraft produced to commercial standards only. These are comparable 
with drawings produced under Form 3 requirements of MIL-D-1000. A direct 
comparison is thus available between the costs to produce these drawings, 
and the costs to re -draw them to conform with Form 1 requirements. LMSC 
performed the Form 1 drawing production task, all drawings being inked, 
and rendered on drawing cloth, and in some cases on Mylar, 
o The drawings produced on plastic film were used as masters for printed 
circuits associated with the Might Viewing Aerial Periscope (MAP). 
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Cost Impact of Imposing Form 2 ; The majority of drawing packages produced for 
space vehicle programs by LMSC, have been prepared to the standards required 
by MIL-D-1QQ0 Form 2 . As stated earlier in this E.M. Form 2 requires that 
I1IL -STD-100 A govern the production of drawings and lists, to the extent specified 
by KIL-D-100Q . Practically, this infers that most of the requirements of HIL-STD - 
100A are forgone, but that the quality assurance activity required to prove that 
compliance with the requirements has been effected is still in force. Figure 2 
shows the complexity associated with this particular Form, versus that associated 
with Forms 1 & 3 . Figure It contrasts the costs of drawings, and the quality 
assurance activity involved in their production for a number of programs 'under- 
taken by LMSC over the last 10 years. It must be noted that LMSC is an aerospace 
contractor, and as such produces drawings under the Form 3 class which are very 
nearly identical with those produced under the Form 2 class. The major differences 
are within the categories required to be checked (see figure 2 ). Similarly, 

LMSC drawings produced for purely commercial programs closely resemble the Form 3 
items . 

Cost Drivers, Forms 1 & 2 : One of the major contributors to the cost of drawings 
produced in compliance with Forms 1 & 2 of MIL-D-10QQ , is the checking function. 
Drawings produced to both these forms are subjected to checking by a drawing 
checking department/group whose members seek to eliminate inaccuracies from the 
drwaings, such as improper dimensions, symbols, misnumbering, wrong call-outs, 
poor drafting techniques, etc. After the checks- function is completed the Quality 
Assurance drawing inspectors review the drawings to determine that the drawing and 
checking functions are indeed completed in accordance with the required standards. 
Prior to final release of these drawings for use by either the manufacturing proces 
or the Government, a further quality check is performed by the resident Government 
Agency Quality Assurance inspector. In-accuracies, non-compliances, or other 

deficiencies are then rectified to Governmental satisfaction, and again, each dwg 
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i'JlNiiO , 


1° Sl ' 3js CTED TO YST ANOT HER SCRUTINY PRIOR TO ACCEPTANCE OF THE DaA'..^.^^ . 

IZ haS Pr0Ved 1 ' :lp0SEit - e t0 determine additional costs to a program nhich arise 
from this practice, but it is obvious that in terms of schedule delays, the 
additional checking functions are both costly, and to a large extent unnecessary. 
Echelons of checkers add little to the drawing value, but much to its costs, 
and it is suggested that this practice be discontinued. 

Conclusions 1 Recommendations: Drawing practices in use throughout the aerospace 

industry tend to be very similar contractor to contractor. Expressed in other terms 
contractors deriving a very high proportion of their business from Government 
Agencies have internal systems for drawing production which conform well with 
the requirements of MIL-STD-100A. In general, NASA requires drawings to be 
furnished in conformance with the requirements of MIL-D-1000 Form 2, but in 
actuality the deliverable drawing is much more likely to conform with Form 3 
after all waivers and deviations granted have been incorporated into a contract. 
Unless NASA intends to purchase drawings and hardware from, commercial sources 
not presently purveying drawings produced to Government standards, it seems 
unnecessary to impose MIL-D-1000 Form 2 on the aerospace industry. On the other 
hand, if Form 3 is imposed, there is a minor conflict in that the Quality Provisions 
of MH-Q-9858A are still required, and this document is not used by NASA, which 
employs its own NKB.5300.1r(lB) . 

As^has been stated in this E.M., the . most feasible approach to drawing control 
wciuld be for NASA to discontinue the use of MIL-D-1000, and issue a simplified 
l document which could be prepared to incorporate only those drawing controls 
essential for production of NASA drawings.: If this step were taken, NASA would then 
be a Government Agency using a drawing control document not used by other Govern- 
ment agencies. To obviate this condition, a feasible solution would be for NASA 
to originate a vexy simple drawing; control document, in cooperation with all 
other Government Agencies. The aerospace industry could be involved in the product- 
ion of this document, which when released, would supersede the present MIL-D-1000. 
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There are several compelling reasons for issuance of a new specification to 
replace MIL-D-1000. These are: 

o Average costs to produce drawings to MIL-D-1000 Form 1 from the data 
shorn in Fig h are approx 11.7$ of Total Program Costs (TPC). Average 
costs of dischargeing the quality assurance functions for these dwgs 
is TPC or k7% of the overall dwg costs , 

o Costs to produce dwgs to MIL-D-1000 Form 2 requirements average 9 
of total program costs, with the assurance function averaging Ij.OpTPC 
or approx li3$ of the dwg production costs 
o Compliance with the minimal requirements imposed by MIL-D-1000 Form 3 
averages 6.h% of TPC , and the assurance functions average 1.$$ of TPC 
or approx 23$ of the costs to produce such drawings . Mere the safeguardr- 
ing of classified information, identification of rights to data, and 
the !UL-Q-9858A provisions removed, which is all that this Form imposes, 
contractors would be employing their own standards of quality assurance. 

This would reduce the costs of the assurance function to approx 10$ of 
overall dwg costs (See Figure U for some commercial programs used as examples), 
o In addition to these values cited above, the benefit of reducing coniusion 
induced by the large number of documents KI1-D-1000 Forms 1 &2 impose, 
would be realized. Reading time for personnel concerned would be greatly 
reduced as another benefit. 

o Since HlL-D-1000 and its attendant document HIL-STD-100A were first prepared 
and released, reproduction methods used to obtain legible copies of drawings 
have improved considerably. This advance in technique obviates the main 
need for Forms 112. 

It is strongly recommended that MIL-D-1000 be rescinded as soon as possible, 
and replaced with a more usable document. 
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MAJOR PREPARATION REQUIREMENTS 

; DRAWING formats per mil-std-ioo 
! drawing practices per mil-std-ioo 
' dimensioning per mil-std-ioo 
numbering and coding per mil-std-ioo 
(includes reidentification rules) 

DRAWING TITLES PER MIL-STD-IOO 
: REVISIONS PER MIL-STD-IOO 
; TYPES OF DRAWINGS PER MIL-STD-IOO 
| ASSOCIATED LISTS PER MIL-STD-IOO 
; ABBREVIATIONS AND SYMBOLS PER MIL-STD-IOO 
USE OF MIL SPECS; AND STDS 
, ALTERED OR SELECTED ITEM DWGS 
; MATERIALS FOR DRAWINGS 
I SAFEGUARDING CLASSIFIED INFORMATION ✓ 

IDENTIFICATION OF RIGHTS TO DATA / 

ORDER OF PRECEDENCE OF SPECS AND STDS PER MIL-STD-IU3 
INSPECTION AND TEST REQUIREMENTS 
IDENTIFICATION MARKING REQUIREMENTS 

LEGIBILITY AND REPRODUCIBILITY PER MIL-D-5U8g/MIL-M- 9868 
QUALITY ASSURANCE PROVISIONS PER MIL-Q-9858 


FORM I 


FORM 2 


R (CONTROL DWGS ONLY) 


M = MILITARY SPECS 


C = CONTRACTOR'S NORMAL PRACTICES 


Fig. 2. Comparison of the Three (3) Forms Cited by MIL-D-IOOO 
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COST BREAKDOWN YQ-3A 


Program Cost Element 

Percent total Program Costs 

Program Management 

3.5 

Engineering Design 

23.5 : 

System Engineering & Integration 

12.5 

Manufacturing 

17.0 

Tooling 

h.2 

Test (including Airworthiness) 

9.9 

Ground Support Equipment 

5.5 

Product Assurance (Rel, Safety, Q.E* & Inspection) 

11.7 

Technical Publications (Including A/C manuals) 

5.2 J 

Field Support (Foreign). 

7.0 

— 

100.0% 


Sub-Breakdown 

Overall Engineering Costs Total ^6 0 

Engineering & Production Dwgs to MIL-D-1000 Form 1. Total » f l0.9 } (a) 

Drawing Inspection & Verification by Q.A. U.9 (b) 

lfote:| Item (a) includes item (b). Item (b) represents approx h$% of item (a) 


j^-o* 3 • Cost Bp 9 3.1cd o yjn If Q— 3A 
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Potential Cost Benefits Derivable from Changes 
to 10 Selected Cost Driving Specifications. 

EM NO: LCPP-3 
REF: 

date: January Di.1975 

H K Burbridge 

APPROVAL: 

engineering /y. 

SYSTEM ENGRG D 

L Hannaford // 


— * — -V V .Ml 

terms of their requirements, and the degree to which program costs are affected by 
invocation of these specifications. Each specification selected was examined to det- 
ermine the cost driving aspects of its requirements, and these cost drivers were listed 


together with a brief resume of the specification's subject matter* 

A task in this study has been the selection of 10 of the original 20 specifications, 
the prescription of changes whose adoption would mitigate the costs each specification 
causes when invoked, and the estimation of cost savings which might arise from applying 
the modified specifications to two typical space programs. 

For ease of reference, the tabulation of the 20 cost driving specifications is included 
in this E.M. and forms pages. 2-6 inclusive. The 10 examples selected for study in terms 
of program cost benefits arising from cost reducing revisions to their requirements 
are listed below as follows: 


( 1 ) 

NHB 5300.M1A) 

Reliability Program Requirements 

( 2 ) 

NHB 5300.1* ( IB) 

Quality Program Requirements 

(3) 

NHB 5300. h( 1C) 

Inspection Requirements 

U) 

NHB 5300. i*(3A) 

Rqmts for Soldered Electrical Connections 

(5) 

NHB 6000. 1(1A) 

Packaging Requirements 

( 6 ) 

MIL-STD-883 

Test Methods 

(7) 

NPC 500.1 

Configuration Management 

m 

MIL-STD-810B 

Environmental Test Methods 

(9) 

MIL-I-8500 

Interchangeability Requirements 

( 10 ) 

MIL-D-1000 

Drawings Engineering, and Associated Lists 


(The prefix NHB denotes NASA handbook and is a compilation of several specifications 
regulating various aspects of a major discipline. For example, the $300.h series 
incorporates those specif cations dealing with the discipline of Product Assurance. ) 
Assumptions : Three assumptions were necessary to permit this cursory study to be 
undertaken: 


(i) Imposition of a specification upon the two typical programs was for 
the entire spectrum of requirements. Frequently space programs have 
been conducted with either parts of a given specification invoked, 
or the spectrum of requirements has been subjected to modification 
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Ho liability 
! Program 


NHB 5300. A (lB) 


nhb 5300 . 4( 1C) 


Quality 


Program 


NHB 5300. 4(3A) 




Inspection 
(•Program) 
System Pro- 
visions for 
Aeronautical/ 
Space System 
Materials , 
Parts Com- 
ponents and 
Service 


( Soldering ) 
Requirements 
for soldered 
electrical 
connections . 


th ° suability' 

P £ ■ 7 irq « ra,n Management ’ 

U) Reliability Engineering 

(3) Reliability Testing ft Evaluation. 
iSffiianentati™: trough contract .tatw.nl or SOW 

o Retires a High Del Parts p rOBram 
° Requires Reliability Demonstration 
o Requires elaborate failure 
investigation 

o Requires program plan ft reporting 


Defines: Requirements in 

(1) Management 

( 2 ) Design ft Development 
13) Data Retrieval. 

(*0 Procurement 

(5) Fabrication 

(6) Inspection ft Tests 


-the .following areas:...., 

(7) Materials ft Non- 
Conforming Articles 

(8) Metrology 

(9) Stamp Control 

(10) Packaging 

(11) Sample Planning 

(12) Government Property 
Control 

I mpiementatio n. trough contract statement, or SOW 
storage^* *£!£& 

Inspection systems. qUlrements •” Provided for the 

Cost Drivers (lB ft ic) : 

a * ^ ua lity Program requires: 

0 Complex multi-levels MRB activity 
o Costly GFP ivipy 

® Process Doc. Review 
0 Sampling plans . •• 

Inspecti on Program requires: 
o Multi-tier inspection 
o 0C Curves, tight AQLs 

o. Elaborate Inspection training 1 certification 
o Complex check & balance system " 

Forml re P°rting system, many document levels 

Defines : Requirements pertinent +n +v> D + 7 V : 

certification of personnel; facilities & gaining and 
materials, processes & procedures and ^ quipment ^ 
type criteria considered essential t ? Pr ° gram - 

quality soldered , 

Crit6ria for ac ceptance/rejecti 0 n^"of 65 

soldered connections, by use of -mi J ctl °^ 01 
as guidelines for visual inspecl&S? 
on contractor documentation & control ih a ; . rgely 
with the cited provisions of the S. aCC ° rdance 


FIG. 1-a 


TWENTY COST-DRIVING SPECIFICATIONS (Sheet 


1 of 5) 
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NHB 5300.i»(3A) 


Continued. . . 


NHB 6000. 1(1A) 


Packaging 


Dr ^ vera : Requires : 

o On-going tz'aining program 

0 •*»* at 

tion process to ^ I ? qUlre lmother se ^™te Inspec- 
c Is Invoked roSiJelv o?"f? ° f i ‘ n ° ther 
applicable or no? ^ lr ° quent baslG »teth B r 


Requi remehts 
for Packaging,, 
Handling and 

Transportation 
for Aeronauti- 
cal & Space 
Systems , Equip- 
ment & 
Associated 
Components 


xMSPC PROC I51A 


Contamination 


for preservation S ra S v fU ^ th the /ieneral requirements 
handling + 1 P ac ^ a C ln 8^ packing, marking, 

dandling, transportation, and related ? , 

“dY 5 uet 1 " 6 xl 

tut does not supersede detaile^s^ciXicaf 

«-«»• ™ 

^Siifl® : or ^S Ugh Progr “" Pi«ctives, contract 

ifirrrSfe tam S e Y f fr uork of the 


MSPC-SPEC-164A 


MSFC-STD-246A 


Control 

and 

Environmental 
protection of 
Space launch 
Vehicles, Space, 
craft. Experi- 
ments, and 
associated 
equipment pro- 
cedures for: 

Directs Compo- 
nent Cleanli- 
ness as per 
151A call outs 


operations? P level “leaning through launch 
Implementation : Contract statement, SOW, or CEI. 


<3ften~when nit needed^ 6 13 invoke <i routinely, 
o Calls out more cleaning than is necessary 
hSd?a?f C °” PleX ° leaning tZ simple 



Control Areas. 
Design & Oper- 
ational Criteria 
of controlled 
Environment 
areas 


£££ ?o n uSneT to — *“<* is 

oontaIns i-tr. 

classes. hsTYfaSed Z SS f* 
operating personnel as well as the roomf speS?LYo„s 

I^ lemen tation: Contract Statement or SOW - 

referenced to the NHB 5300.4 series. S 

out \ ^higher ' clfsfo^roo, that WASA calls 

imposes cleanliness requirements 
which are unnecesss-ri it. , environment 


FIG. 1-b 


saril^i stringent 


WENTY COST-DRIVING SPECIFICATIONS (Sheet 2) 
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MAC5A/ MIL // 


MIL-rJTD-883 


SUBJECT 


Test Methods 


SYNOPSIS 


Defines & Specifies : 20 specific test methods for 

high- reliability piece ijarts, ranging from method 
1005D which treats the determination of operating 
life to 20.12 which specifies the methods and dosage 
levels for X Ray inspection. This document governs 
the entire conduct of a HI-Rel parts program for IC 
and semiconductor devices as well as IGT/MSI circuitry 


| MM 8040. 12 
NPC 500.1 




MSFC Contractor 

Configuration 

Management 


IMIL-STD-100A 


| MIL- D- 1000 




Engineering 

Drawing 

Practice 


Implementation : Contract statement, SCW. Requires 


Applications plan as CEI, also must be cross refer- 
enced with the NHB 5300.4(lA) reliability program, 
and the data from this std. form part of the relia- 
bility demonstration plan when such is specified. 


Cost Drivers : Low parts yield after all tests -are 

completed, often only 6-8$ 


o Reliability parts failure rate 

o Reliability Demonstration to hi'gh confidence level 
0 Very costly test equipment 
o 100$ parts screening and burn-iii 


Defines: Specific methods required by MSFC which, 


when used by contractors afford effective control of 
hardware configuration. Includes elaborate trace- 
ability procedures, batch control, manufacturing 
controls etc. 


Implementatl on : Contract Statement, SCW, Plan is 


usually a CEI. 
Cost Drivers: 


There are 5 different configuration 


management- documents in use at NASA, and in seme 
points they contradict one another, 
o Whichever document is used seems to be at the 
discretion of the program office, 
o Waivers and deviations are permitted by formal 
request/approval, 

o Configuration control is imposed routinely whether 
or not it is necessary to control the hardware. 


These two documents are applied in conjunction with 
each other. The STD calls out procedures and formats 
governing how the drawings should be produced. MIL- 
D-1000 ‘calls out how many and what type of drawings 
should be produced, sufficient only for the job. 

There are 3 forms associated with this spec. Forms 
1,2,3. Forms 1&2 are formal regimens with rigid 
requirements to be followed. Form 3 allows con- 
tractors to use their own system essentially, and 
is much less costly to implement. Forms 1&2 inter- 
face, reference, and apply 35 other documents when 
used. Form 3 does not. 

Cost Drivers : Cost drivers have been treated else- 

where in this study report (para. 4.4.9). 


FIG. 1-c 


TWENTY COST-DRIVING SPECIFICATIONS (Sheet 3 ) 
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NASA/MIL # 

SUBJECT 

SYNOPSIS 

MIL-STD-461A 

MIL-STD-462 

KMC- Require- 
ments for 
EMC-Measurement 
of 

These two documents are used in conjunction with each 
other. Respectively they define: 

The requirements & test limits for measurement and 
determination of Electro Magnetic Interference char- 
acteristics (emission & susceptibility) of electronic, 
electrical, and electromechanical equipment. 

The detailed techniques to be used for measurement 
and determination of EMI characteristics. 

Implementation (both): SCW, CEI, or Equipment Spec. 

Cost Drivers : 

0 Additional Test Equipment costs 
0 Additional Test Program Costs 
0 Additional Design Costs .to minimize EMI 
0 Additional cabling routing and bonding costs 
0 Additional Facility Costs if magnetic cleanliness 
is required, as well as additional material and 
device costs. 

Standards are frequently invoked 'routinely where it 
is suspected that EMI MAY be a problem. Need for 
invocation should be determined by early Design Review 

| MIL-E-6051D(l) 

} ... 

Electromagnetic 

Compatibility 

Often invocation of MIL-E-6051D(l) is sufficient as 
this sets forth general Electromagnetic Compatibility 

Requirements 

requirements . These can be met often without the 
46lA, 462 invocation. 

>■ 1 — 

1 

} MIL- STD-499 

f 

i 

, 

1 

t 

1 ~ 

'.I..'- 

• 

Systems Engrgs 
Management 

' . . . . . 

Defines: Requirements for the planning, execution 

and management of a fully integrated system engineer- 
ing effort. The elements of a management plan are 
defined. The scope and elements of a system engineer- 
ing effort are defined and illustrated, showing the 
kinds of analyses and reviews necessary to a fully 
integrated effort, and the means of assuring con- 
tractual compliance. This document was developed by 
USAF Systems Command on the basis of experience with 
the use of the AFSC series of manuals on systems 
engineering, and is a replacement for the 375“ 5 
systems engineering management procedures. MIL-STD- 
499 is intended to set forth all functions needed to 
plan, organize, and operate a systems engineering 
effort, FOR TRIAL PURPOSES... The intention is that 
the document ’is a guide, and would be applicable to 
any contract leading from concept formulation thru 
hardware design & development.; Application to NASA 
programs is limited by the orientation to DoD methods 
of operation, and DoD and NASA methods, while similar 
in concept, differ in detail. Application is limited 
even on DoD programs by Notice 1 indicating that 
MIL- STD-499 is for trial purposes only. The document 
is not fully developed, and will be amplified further 
by DoD. i 


FIG. 1-d TWENTY COST-DRIVING SPECIFICATIONS (Sheet 4) 
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nasa/mil # 

. — 

SUBJECT 

SYNOPSIS 

n 



i4IL-STD-^99 

Continued. . . . 

Implementation: Through SOW. Note: Despite the 

. 


trial nature of this std., as cited above,. it is 
invoked routinely with no attention directed to 


• 


Notice 1. The CIRL accompanying contracts which 


• • 


call out this standard, call for the full document- j 

ation complement that the s tandard recommends. 

Cost Drivers: Implementation of this STD reuuires: 

' . 

. 

. ‘ • 


0 Generation of full WBS by Task, Manning, arid \ 

task segments . j 

0 RAS (rqmts. analysis sheets) must be made for each 
WBS item to 7th level 

0 PERT or other tracking techniques are required. 

0 Cost reporting by NHB 9501.2, for both contractors 
and NASA centers 

0 Full Change control, configuration control with \ 

flow diagrams, and numerical tracking are called out. 5 

0 Monthly, qtly, semi-annual, & annual progress reports 
with financial status are required, as well as , 

periodic program audits. 

Estimated Cost Impact: Where this STD is implemented,' 




costs of documentation and general program control 
increase by 30 -^ 0 $. 


MIL-STD-81QB 

Environmental 

This military standard is a compilation of standard \ 

o 

. 

' 

: . : ■ ... 

. ' 

. " , . 

Test Methods 

. . 

• . 

- '. ■ - 

test methods for determining the resistance of equip- 
ment & materials to natural & induced environments. 

Test methods cover such areas as altitude, high or 
low temperature, rain, dust, shock, vibration, and 
various combinations of conditions. Methods & base 
conditions for each test are specified. Each method 
lists details which must be specified in the equip- 
ment specification. 

Implementation: Contract, SOW or CEI. 1 




Cost Drivers: 


■ 

9 


0 Some of the standard tests specified are costly in 
terms of time and equipment to implement; i.e., 
thermal vacuum test at the vehicle level. Frequently ■ 
full vehicle tests although called out are unnecessary! 

0 This STD causes specification proliferation, in 

terms of the other specs it calls out. Examples are: 
MIL-STD-202D, Test Methods for Electronic and Elec- 
trical-piece parts, MSFC-SPEC- 101 B Flammability, m 

MSFC-STD -366 Penetrant Inspection method. MSFC- 
SPEC-259A Radiographic Inspection methods, etc. 

The document should be called out with the qualifier - 1 

"To the extent applicable" rather than routinely, or 1 

applied with a list of exceptions. * 


MIL- I- 85 OO 

Interchange- 

Defines: Interchangeability of parts and components. m 



ability 

Cost Drivers: Often applied to programs where very 1 


* /, 

Requirements 

small quantity of vehicles does not require detail 1 

of interchangeability. j 


FIG. 1-e TWENTY COST-DRIVING SPECIFICATIONS (Sheet 5) 
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during the contract negotiation process. 

(2) Cost savings estimated to accrue as outcomes of modifying the 10 
specifications studied represent only reductions in man-hours 
expended during the course of the programs studied. No materials, or 
hardware cost savings have been considered. 

(3) Changes proposed for the 10 specifications are those which have been 
considered to provide a Mandatory Minimum set of requirements. 


Program Selection : The two programs selected fear the application of the 

low cost versions of the specifications are : 

(1) Mariner Venus Mercury 1973 (MVM-73 • JPL) (Contr, Boeing Corp) 

(2) Atmospheric Explorer (GSFC)(Contr. RCA Corp Astro-Electronics Divn.) 
Approach ; The approach was to identify the percentage of total program costs 
attributable to each of the 10 cost driving specifications, for each program 
selected. This process was then -iterated, but the assumption was made that each 
cost driving specification had been modified to a mandatory-minimum version. 

The costs of applying such a specification were estimated, again expressed as 
percentages of total program costs. Two summations were then compiled of actual 
costs, and estimated costs. The difference between these two summations represents 

to potential cost savings realizable for similar space programs, should specifications 
be restructured/modified with mandatory minimum costs as a governing criterion. 

The results of this exercise, together with the cost modifying changes recommended 
for application to the specifications are listed as depicted in Figure 2, Page 8 


Limitations : Neither of the programs chosen invokes the specifications selected 
in their actual documented form. Both MVM-73 > and AE have their own program specs 
written and imposed. The NASA centers (JPL, and GSFC) wrote the program specific 
ations based upon the 10 selected examples. Study of each specification versus 
the applicable portions of the center-prepared program documents revealed that 
in both cases, all the major cost-drivers were included. Neither the Pioneer- 
Venus, nor the Nimbus G program was selected for study, as both of these had 
less visibility into the cost data. In the case of the HEGS program, this is 
on-going, and definitive cost data are not available. The P7X-2 program is not 
a NASA program, and thus does not invoke the same set of specifications; i.e. 
MIL-Q-9858A is used for the Quality Assurance Program regulation and control 
functions. In the light of these limitations, the exercise should not be considered 
as highly definitive, but rather as cost savings trends indicative. 
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GOST-DRIVING 

SPECIFICATION 

INVOKED/APPLIED 

\ 

MVM-73 


1 — 

_ ALL VALUES LISTED ARE IN PERCENT TPC* 

- 


WMM 

COSTS 

REPORT 

SAVINGS 
t KST »T 

MAJOR COST REDUCING CHANGE APPLIED (ASSUMED) 


NHB 5300.1|(1A) Rel. 

1 o 

Hi 

1.3 

38j/ 

•/.8 

Eliminate Parts Population Prediction. Use dynami 
simulation. Eliminate Reliability Demonstration. 

<: 

NHB 5300.I»(1B) Qual 

2.5 

52/^ 

"1.2 

2.2 

55j/ 

XL.0 

Delete Formal MRB Rqmt. Substitute informal MRB. 
Simplify Mgmt System. Reduce Reporting. Delete 
O.C. Curves. Simplify Inspection Instructions. 

NHB 5300.1, (1C) Insp 

3.8 


R9 

35|/ 

/2.0 

Inspect at Highest Possible Level. Use Double 
Sampling. Minimize Insp'n points in process. 

HIliRli 


U6#/ 

■ 

gH 

Eliminate NASA certif'n for contractor solderers 
& inspectors. Approve process/training only. 


NHB 6000.1(1A) Pkg'f 

bps 

33/^ 

Xo.l 

0.2 

25#/ 

/.15 

Eliminate this General Specification. Detail 
Rqmts in S.O.W. per mandatorv mimmiun 

Test 

MIL-STD-883 Methods 

M 

EcIQH 

_7» 3 


Limit Life Testing. Limit Rqmts for XRay Tests. 
Limit Test Repetitions. Specify Mimima not Max 1 a. 

Config'n 
NPC. 500.1 Mgmt. 

0.35 

1*3*/ 

/"0.2 

0.8 

25#/ 

/.6 

Eliminate Batch Traceability. Elim. Mat'l Certs. 
Simplify numbering & Codes. AdpIv onlv f m* ^ * s /r 

Envir 1 1 

MIL-STD-810B Test * 

3.2 

56/"" 

/.u 

3.9 

62 #/ 

/5 

Eliminate Thermal Vac at syst level. Apply at 

lvl. Minimize Test Repetitions. Limit duration 

Interch'ly 

MIL-I-8500 

0.2 

5o#/ 

/0.1 

0.25 

ho %y 

/.15 

Limit to Programs with several S/C. Std'ze 
parts & components. Use Std Designs as possible. 

Eng Dwgs 
MIL-D-1000 & Lists 

6.8 

i*o#/ 
/.08 | 

6.60 

1*9#/ 

'""3.96 

Reduce Form call out frm Form 2 to Form 3. 
Limit No. of Copies. Simplify QA & Inso Romts 

Spec Oontrib'n to 
Program Costs 

27.50 

17.23 

27.75 

17.11 ! 

/j\> Percent Savings due to Changes 
& New, Reduced Percentage of TPC. 

TPC 

Potential Svga. ^ 

-1Q-.3# 

$3.9M 

10.6). 

S2..3liM 


* TPC = Total Program Costs 


Fig. 2. Affect on Program Costs of Specification Changes 
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Co st Estimation Method : Wherever possible, dollar values as reported by the 

two prograns were used as the bases against which to estimate savings potentially 
available if specifoations were changed as tabulated in Figure 2. In many cases 
the dollar expenditures reported did not detail the tasks perfonned for such 
amounts. For instance, the cost value shown for MVM-73 for Quality Assurance did 
no break out the Quality Engineering tasks separately from the Inspection tasks, 
in this case, the assumption was made that Inspection averages approx 60 % of the 
overall Quality Assurance expanse to a program. This is an WSC factor, which 
experience has shown applies fairly well to most aerospace contractus. Within 
e Quality Engineering task the work does not vary too much in terns of specific 
task, program to program, or contractor to contractor. The assumption was made 
at the usual tasks were performed such as Trouble >, Failure Reporting, KRB 
activity (Material Review Board) Preparation of Inspection Instructions, Purchase 
er eview, etc etc Similarly for the Inspection routines and associated tasks, 
e assumption was made on the basis of what HSC would have done had the contract 
been performed by the company, as well as a count of the .inspeotable hardware items 

“ °° S ^ faCt ° rS are used t0 ^lar contribution of manhours per task, 

us if a task is deleted, the gross manhours estimated for the cost of that task 

is reduced accordingly, and the reduced amount is converted to program dollars. 

A new, reduced percentage of total program costs is then computed. 
a giLights ^Conclusions ; Several interesting points emerge from this exercise: 
U) In the initial ISP? study LMSC estimated that specifoations could 
drive costs of a program over a range of 25-1,0 *TPC. It would appear that in 

the case of MVM-73, and Atmosphere Explorer programs that both exhibit a spec- 
lfications cost impact within this range, (ref Fig. 2) 

(2) Accompanying the estimate cited in (l) another estimate was made 
a use of cost effective, mandatory minimum specifications could afford a 

reduction in specification impact ranging from 10-16 *TPC. Again, this estimate 
wo appear to be valid for both programs considered . 1 (ref Fig. 2) 

(3) Although all 10 specifications listed and examined are potential 
cost drivers, dependent upon how invoked and interpreted, the major cost driving' 

specs are those controlling Product Assurance activity within the programs, and 
those controlling Test Activity. 

(i *) Product Assurance Activity accounts for 6.9 /oTPC on the MVM -73 
Program, and i$.J% on the AE Program. While these values are average for soace 
programs of this type, they can be reduced appreciably' by use of specs whose 
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requirements are based upon mandatory minimum approaches. (Fig. 2) 

(5) In the previous LCPP program LMSC expressed the opinion that the 
major cost contributors to programs, other than the basic design engineering, 
system engineering, and manufacturing functions were Test Operations, & Product 
Assurance Operations. Examination of these two areas for MVH-73 and AE programs 
appear to support this contention. It is in these two areas that the greatest 
cost impact of specifications is also realized. 

(6) Both the N7M-73 Program, and the AE Program are considered within 
the aerospace industry to be Low Cost Programs, whose specifications are defin- 
itive and rigorous. Yet significant cost benefits could accrue in similar future 
programs, if the major cost driving specifications imposed are revised to a 
mandatory minimum condition. 

Conclusion ; The specification library in use by NASA and the DoD afforded an 
effective system of checks and balances during the decade of "Success At Any 
Price". Such a system of checks and balances is too costly presently, and will 
be even more costly in the immediate future, when the "More Space Program for 
a Dollar Spent" aphorism is paramount. A new library of specifications based 
upon what is the least that will do the job, is needed urgently, and is fast 
becoming overdue. 
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Introduction ; The NASA Low Cost Systems Office imposed this task as an attempt to 
determine the degree to which programs undertaken by the various NASA centers impose 
the same specifications. A representative cross section of NASA space vehicle programs 
wals supplied for the study, these being selected by the LCSO. The programs were 5 
in number from NASA centers with a USAF program contracted to LMSC -included to serve 
as a check point for commonalities and differences DoD versus NASA. The programs 
studied were as follows; 


PROGRAM 

CENTER 

CONTRACTOR. 

NIMBUS G j 

: 

GSFC 

GENERAL ELECTRIC COMPANY. \ 

ATMOSPHERE EXPLORER 

GSFC 

RCA CORPORATION. 

HEAO A/C 

MSFC 

T HOMPSON-RAMO-WOOLBRIDGE CORP. 

PIONEER-VENUS 

ARC 

HUGHES AIRCRAFT CORP. 

MVM-73 

JPL 

BOEING CORPORATION 

P71-2 

USAF 

LOCKHEED MISSILES & SPACE COMPANY 
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Task Categories & Definitions ; To better scope the task specifications were segregated 
into several categories. This approach has the advantage of highlighting similarities 
existing among specifications, but has also the marked disadvantage that many specif- 
ications must be classified in more than one category. The categories chosen may be 
listed as follows, and dual category examples are also given; 

o Materials Specifications Tells what to use 

o Process Specifications.... Tells How to use, how to make it. 

o Requirements Specifications Customer (NASA center) tells contractor 

"Thou Shalt" 
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o Performance Specifications....... NASA tells contractor hardware 

at all levels shall perform/be 
operated within, given parametric 
ranges & limits. 

o Assurance Specifications......... Tell HOW the product is to be 

tested. 

Tell How to conduct test 
Tell the STANDARD product must 
meet, over what period of time. 
Tell the DEGREE of proof NASA 
requires. 

As has been stated, many of the specifications cross category lines, and the task of 
assessment and relegation of a specification to a clear cut category is at best one 
of engineering judgement, at worst almost impossible. Some examples are: 

o MIL-C-17D. . .RF Coaxial Cables. Gives requirements, but also tells 

how to test, & calls out material, 
o NHB 5300. U(3A) Soldering. Tells how to solder, gives requirements, 

gives tests & examinations to assure 
requirements are met. 

o MIL-HDBK-17 Plastics for Flight Vehicles: Tells what to use, how 

to use it, how to test, 
to what performance req- 
irements, in what manner 

To the degree possible, the specifications studied during the course of this task 
were grouped by the most applicable single category. 
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Task Approach ; The approach taken was to construct a matrix of specifications 
for the 6 programs selected. This matrix, which forms Figure 1 attached to this E.M. 
does not include, except by count, documents which may be classified as performance 
specifications. There are two reasons for not detailing these specifications: 

(1) There is little relationship, program -to -program among these 
specifications, which are, by nature, program peculiar. 

(2) The sheer quantity of such specifications would make the matrix too 
unwieldy to handle, and impair visibility for the other specification 
categories treated. 

Program Practice Identifiable from the Matrix : The matrix indicates clearly that 
therp is very little uniformity of procedure NASA center, to NASA center, in the 
generation, preparation, handling, and imposition of specifications in any category. 
Several practices are identifiable however: ' 

(1) 3 centers, GSFC, ARC, and JPL, write, and issue materials and process 
specifications as program peculiar documents, making no use of exist- 
ing documents in those categories. Occasionally these latter documents 
are used as models. 

(2) All centers, and the USAF, without exception issue highly detailed 
program peculiar performance specifications at all hardware/software 
levels within a program. This practices is repeated for every program. 

(3) If a given center believes such are necessary it will prepare and issue 
center peculiar/program peculiar assurance specifications, despite the 

fact that adequate NASA-wide examples of these documents exist. 

(i|.) At least two centers, ARC and JPL, write an all-encompassing program 
document in which all specifications, in all categories are amalgamated 
into a polyglot "amorphous mass". JPL has implemented this practice 
on several programs in addition to the MVM-73; e.g. MM-6? and others 
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of that on-going series. 

(5) While the practice is not identifiable from the matrix, the HEAO 
program from MSFC has caused the program contractor to write the 
hardware performance specifications, MSFC reviewing and approving only. 
Cost Implications from the Matrix : The degree to which program costs have been driven 

by the practices listed is difficult to assess with accuracy. There are trends however, 
and inferences which can be made. Tabulated below are what are termed Assurance Costs 
for 8 programs, where Assurance is defined to mean activity undertaken by programs to 
assure that reliability, maintainability, and quality requirements have been met: 


Program 

Contractor/ Center 

Assurance Costs #TPC. 

MVM-73 

Boeing/ JPL 

6.1 

AE 

RCA/GSFC 

3.1 

Pi o. Venus 

hac/arc 

5.5 (est. prog current) 

Nimbus G 

gec/gsfc 

6.2 

HEAO 

trw/msfc 

6.0 (est. prog current) 

P71-2 

LMSC/USAF 

5.1 

OSO-I 

HAC/GSFC 

5.6 

ATS-F 

Fairchild/GSFC 
LMSC Flex-Rib 
antenna. 

8.U 


The ,two last programs shown in the tabulation have been included as HEAO and Pioneer 
Venus are on-going current programs whose cost data are not available to LMSC, and 
whose cost percentages are estimates only based on discussions with ARC, MSFC person- 
nel concerned with the assurance disciplines. These estimates may not be too realistic 
when the programs comes to completion. 
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An appreciable difference in percentage of total program costs is apparent from 
a review of the listings . The programs ,with the exception of the Atmosphere 
Explorer, average about 6% TPC , whereas the AE reports only approximately half that 
percentage. The obvious question of what was done differently on the AE program 
may be answered as follows: 

o All the programs listed, with the exception of Nimbus G, and P71-2 
performed the assurance functions prescribed in NHB 5300*U(1B) 

(ref: Matrix) 

o P71-2 was responsive to MIL-Q-9858A (USAF imposed) which is in all 
respects equivalent to NHB 5300.l4(lB). 

o Nimbus G was controlled in the assurance disciplines by GSFC~lj.30-311 
Rev. A. which is a combination specification covering Reliability, 

Quality Assurance, and Configuration Management. As such it is equiv- 
alent in effort expended to NHB 5300.k(lA>; Reliability, NHB 5300«U(1B) 
Quality Assurance, and NPC $00.1 Configuration Management. 

O' AE, in a major departure from the norm,invoked the NHB 530O.it series 
by intent, and as a reference and guide only. The program required the 
contractor, RCA, to prepare and implement an assurance plan which was 
"lean and mean ". This was done. Significant by absence were such items as 
reliability life assessment testing, reliability demonstration, elaborate 
configuration management plans, formal material review board activity, 
and most of the required documentation usual with assurance program 
plans. 

Outcomes of this approach were : 

(a) an overall cost reduction for the assurance program 

(b) No loss of quality or mission success 

(c) A significant reduction in documentation to be generated and stored. 

(d) A tightly knit team of center and contractor-personnel, 
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cost conscious, and program success-motivated. 

None of the essential functions within the product assurance discipline was sacrificed 
and the operating plan was implemented with a minimum of lost, time, and the fewest 
personnel engaged at the customer/contractor interface. 

Other Trends: Reference to the Specifications Matrix shows clearly that each MSA 
— — er ten , d? t0 P urs ue its own course when selecting specifications applicable to 
a program. For instance MSFC, in implementing the HEAO program with TRW as contractor 
has issued an extremely complete list of program specifications in the form of an 
Applicable Documents List HEAO Spacecraft Phase C/D. . . HDM-i*222-l dated Feb 7/1971*. 
This document contains not only a large number of center generated specifications of 
the materials and processes type, but also contains Military and Federal Specifications 
Handbooks and Standards. This document is 1*1* pages in length, plus 8 pages of general 
preamble. In addition another document issued Jan l?/7l* titled Contract Items Specif- 
ication for HEAO-B Observatory System. . .72M10067, re-enumerates all the specifications 
of the February document, and furnishes a comprehensive and exhaustive set of perform- 
ance criteria, against which TRW has generated detailed specification documents for 
MSFC approval. It is difficult to accept this practice as cost effective. 

o MVM-73 kas achieved wide acclaim throughout the aerospace industry as a low 
cost, on target, on time program. Despite 35 in-flight anomalies (none of which had 
any major, or long lasting affect upon the successful Outcome of the deep space mis- 
sion), the Boeing Corp was able to earn a total fee of over lh% for its efforts on this 
program. Two thirds of this fee was for cost effective performance within program 
objectives, but while this award was doubtless merited, the agency JPL, might have 
saved even more program costs in the area of specifications, by instituting a differ- 
ent practice. JPL wrote a highly detailed, fully comprehensive program-controlling 
master specification. This document, titled MVM-73 Space Craft Design Criteria 
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first issued December 11/1970 incorporated many of the "How To" types of specifcation, 
as well as calling out materials and processes. Very few MIL SPECs/STDs were called 
out, and very few NASA specifications either. The JPL specified requirements were 
instead incorporated in a document titled Spacecraft System Requirements Document 
615-12 Contract Exhibit III. Further amplification of the tasks to be carried out 
by the contractor were issued as Exhibit IV Part 2. Work Units. In this document, 
al ,1 tasks required were numerically coded against an elaborate WBS, with JPL 
controlling specifications imposed as necessary. In short, the program had its own 
specification hierarchy. This approach appears to have been satisfactory in not only 
the case of MVM-73 but also in the case of several other program with JPL as the 
program center and Boeing Corp as the contractor. The program documentation list 
includes no less than 36 separate implementation plans, with attendant specifications 
controlling performance of hardware and software. While it is difficult to argue with 
program success, it is also difficu3,t to imagine that a more standardized approach to 
program specification would not be possible and cost effective. 

Conclusions and Recommendations : Several conclusions can be drawn from a review of 

the matrix. These are listed as follows: 

(1) Very little commonality exists in specification approaches and 
practices among the NASA field centers. 

(2) Centers tend to preserve an autonomy which is neither cost-effective 
nor indicative of any standard NASA practice or programmatic approach, 

. • i: : ; i ' ! • . : i 1 . , " ' j : : .. ' \ 

(3) In E.Ms #3 &#U in this series on specification practices the cost driv- 
ing aspects of the assurance disciplines have been explored. In the 
case of the AE program a low cost approach was taken to these dis- 
ciplines by implementation of a low cost program plan. 

(U) All centers, tend toward over-specification and over-elaboration 

doubtless as a consequence of the long -prevalent "Success at any Price" 

philosophy in vogue at NASA. 
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(5) The non compliance with any standard approach to specificat- 
ions evidenced at NASA centers applies equally to DoD, as 
indicated by the P71-2 program included in this E.M. for 

' ' . ; i ( [ : . . 

comparison. Levels of cost, in the assurance disciplines, 
and in the number of program peculiar documents required 
by a contract appear to be similar, NASA vs USAF. 

Recommendations : It would appear desirable for several changes to be made in NASA 

specification practices. These would include, but not necessarily be limited to the 
following ; 

o A standard set of specifications should be prepared for use by 
all NASA centers engaged in space programs. (This excludes perf- 
ormance specifications which will vary, but even these could have 
a standard format.) 

o NASA should cease to issue "How To" type specifications, aero- 
space contractors are qualified to employ their own methods. 

Sensible performance parameters, and finite end-results expected.? 
would be preferable. 

o In the assurance disciplines, the NHB $300. b. series stands in need 
of revision. Plans for these disciplines based on mandatory minima 
for programs, perhaps modelled along the lines of those used for 
Atmosphere Explorer would suffice as revisions. Additional weight 
to this recommendation is lent by the current DoD activity to 
materially revise MIL-Q-9858A, a direct equivalent of the NASA 
NHB 530O*Jt(lB), Contractors no longer can be responsive to MIL-Q- 
9858A at a price acceptable to DoD. 

o Specifications released to the aerospace industry are required to 
be NASA & Tri-Service (NASA/DoD joint directive May 19610 . Presently 
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this is not the case. Most NASA specifications issued and invoked 
on contracts are "Center Peculiar'S and are not co-ordinated even 
center to center. 

o A central co-ordinating activity should be set up for the 
generation, control, and release of specifications. The 
overall objective of this activity/office should be to 
inject into the specification process the maximum amount 
of standardization. Cost effectiveness of a specification should 
be considered before its release, and the office should be 
authorized by NASA management, at appropriate levels, to co- 
ordinate all specification activities with a similar DoD 
office, reported to be in existence by June 1975. 


Note: 


L has been considered throughout as though it were a NASA agency. 
Of course, this is for convenience only, as JPL behaves in contract- 
ual matters very much as do the other NASA Field Centers. 
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1. Objective : The underlying objective in establishing a Central Specification 

Control Office (CSCO) is to effect a substantial saving in NASA Program expend- 
itures, Such an end may be achieved through elimination of specification 
proliferation,, duplication, ambiguities, and overall lack of regulation and con- 
trol. In addition to furnishing an improved specification control system, the 
system may be employed to accommodate other program documentation, thus afford- 
ing further savings. In other EMs in this series it has been estimated that 
better control of specifications and program documentation could result in 
savings in the following range : 

Specifications Savings 10-16$ of Program Collars (TPC) 

Documentation Savings........ 8-11$ of Program Dollars (TPC ) 

Total Possible Savings 18-27$ (TPC) 

This EM furnishes planning for such a Control Office, to the degree necessary 
to demonstrate the concept, which if implemented, can afford such cost benefits. 
As better specifications, better control of these program regulating documents, 
and overall improvement in their content in terms of intelligibility, and brevity 
are realized) the synergistic effect will be appreciable as more and better hard- 
ware per program dollar expended. This plan is offered in two parts: Part 1 
depicts the present situation, and Part 2 which treats the institution of the 
CSCO. : 

2. Part 1; The Current Situation : Specifications of a multitude of types, per- 
taining to every phase of development and manufacture of space hardware/sofware 
are available today. NASA can, and usually does, invoke a large number of these 
when contracting with aerospace companies. The companies are vitally concerned 
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with compliance with NASA terms and directives on a continuing basis, and 
therefore spend much effort in assuring that such compliance is as complete 
as possible. Costs of such activity are passed on to the Government in the 
course of contracted performance on the basis of: " NASA can have anything it 
wishes, provided due payment is forthcoming " In the case of specifations, a 
bewildering maze exist through which both NASA program managers, and the contr 
actors must find a path. As with any other maze, there are many dead ends, 

i 

and side avenues residual from the days when the aerospace Industry was learn- 
ing the space business. 'When these side avenues are invoked, either by inad- 
vertence, or habitual practice, costs can only increase. Figure 2.1 illustrate 
the number of specification sources existing, any or all of which can be invok 
ed by NASA at the discretion of a Program Manager or COR. For sake of clarity 
the many cross links are not shown, but it is notable that the only "Escape 
Hatch" is a small one, designated WD to denote the Waiver & Deviation loop. 
All avenues funnel into the MSA Center* Program Office, and again for clarity, 
the contracts loop is omitted, as is the aspect of multiple center involvement 
3 . Current Regulation & Controls : In I96I4 the DoD discerned that the hierarchy 
of military specif cations and documents was growing at an alarming rate. A Reg 
ulating 3 oard was formed, the NASA & Tri -Service Review Board , and vested with 

the authority to: ' > 

o Review existing specs, and require updates, additions, and deletions, 
o Co-ordinate specifications release among the several Govt. Agencies 
0 Furnish advice on an Ad-Hoc basis to both Govt Agencies & the aero- 
space industry, on improvement of the specification system. 

0 Act as mediator, Govt vs industry for specification matters, 
o Fund aerospace contractors to prepare specifications under contracts. 

0 Employ delegates/specialists from both Govt, and Industry. 


Part.l, Figure 2,1 
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The Board is comprised of a chairman, a secretary, two members from each 
of the armed services, two members from DoD, at least one member from each NASA 
field center (7 in all) plus a NASA/KQ member, and six members-at-large from 
the aerospace industry. These latter members are not full time participants 
however. They are consulted as the need arises, and act as intercessionaries, 
aerospace vs the Board. Unfortunately, the board has comparatively little 
affect upon the specification maze for this reason: 

Each of the major contracting agencies, i.e. Army Navy Airforce, and each NASA 
center reserved the right to generate such specifications as each believed to 
be necessary to its own operational peculiarities. For example, a MSFC Procedure 
is an internally- generated document, and there is no requirement for it to be 
co-ordinated among the other NASA centers, NASA KQ and the Review Board. .The 
outcome is that such a document can have, and frequently does have, parallels 
existing in other centers than the one generating it. Each document is autonomous, 
subtly different (sometimes grossly different), and binding upon a contractor 
if invoked. 

3.1 Board Operations : In a non-NASA milieu, the Board is somewhat more effective. 
This is due primarily to the fact that the Armed Services were largely respon- 
sible for the generation of many of the MIL prefaced documents in use today. 

NASA, as a late comer into the aerospace industry domain, also uses many of 
the MIL documents, as in NASA operations often there was no time to generate 
NASA peculiar documents. The converse does not hold good however, as it Is 
very seldom that the Armed Services use a NASA generated document. Since the 
inception of the Board^NASA has gone its own way with respect to generation, 
imposition, and implementation of specifications, and there have been almost no 
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instances recorded when NASA adopted a specification, or other regulatory 
document, reviewed and released by the Board. However, the Board has released 
many documents to the services after review and coordination, most of which 
have borne the statement "NASA A Tri-Service". In almost all cases NASA has 
chosen to ignore this entitlement, although by definition of the Board, NASA 
is required to use an existing document if possible. In short, the Board, so 
far as NASA is concerned, is a "Toothless Tiger". A very good example of -this 
fact is the case of MIL-STD-882 System Safety Program for Systems A Associated 
Equipment: Requirements for . After some 22 months of effort commencing circa 
Jan 1969, the specs A Stds groups of Me Donne 11- Douglas, LMSC, and Hughes Air- 
craft Corp, prepared, revised, and co-ordinated drafts of the STD with all 
services and NASA, the Board acting as intermediary. The completed document 
was released in October 1970, and adopted by the Armed Services, primarily 
USAF. NASA, cl aim ed special safety precautions/measures had proved necessary 

for the manned Apollo program, and would continue to be necessary for the Sky lab 
program. As a result NASA originated the NHB 1710. 1 series dealing with Safety, 
although the MIL STD would have sufficed with but minor revisions/additions. 

3,1,1. The Board Procedures : On the average the Board requires from 18-26 
months to handle a new document, and the method of handling is shown in 
Figure 3.1.1 Specification Processing . As illustrated, the handling mechanism 
is a close-loop complex piece-wise step system, which theoretically can be 
entered at any step point in the chain, so that existing documents can be rev- 
iewed, updated, and re-issued. In practice the system is serial in operation; 
remove a lin k in the chain, and the entire operation is disrupted. Some "short- 
cuts"are possible, by means of pre -determined by-passes, such as allowing the 
aerospace industry to prepare and implement the spec, de-bug it, and then submit 
it to the Board together with recommendations to implement. With Board concurrence 


Part 1. Figure 3.1.1. 
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dissemination then proceeds immediately. As is obvious by the average elapsed 
time, and the fact that NASA pursues its own course, the Board affords little 
advantage to NASA, at least in the space hardware domain. The costs of operat- 
ing the Board are not inconsiderable, although no cost data are available to 
LMSC currently. 

, Re commendation ; LMSC recommends that NASA terminate its participation in 
the Board, instituting instead a Central Specification Control Office as 
described in Part 2 of this EM. 

Part 2 A Central Specification Control Office (CSCO) 

2.1 General Pi arming • It is recommended that the Specification Control Office 
be located at NASA/HQ, with facilities as described herein, and with represen- 
tives at_each NASA Field Center. The Office need not be a large organization, 
and most probably would not require the full-time services of more than b-5> 

persons : 

o A Coordinator/Supervisor, (and clerical support) 
o A Cost-Cost/Worth analyst 
o A Librarian 
o A Senior Technical Writer 

Technical services would be provided on an "As Needed" basis, and the Field Cen- 
ter Representatives would undertake NASA/HQ liaison, and liaison with the aero- 
space industry on a part-time basis, perhaps b0% of their time being employed 
in this manner. The functions of the CSCO are illustrated in Figure 2.1.1. 
Functions of the CSCO, and the aims and objectives of the CSCO are the subject 
of .Figure 2.1.2. titled Aims and Objectives 


Part 2. Figure 2.1.1 
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2.2 Systemic Lo^ic: The logic of the operation is essentially extremely 
simple, and is depicted in Figure 2.2.1. CSCO Systemic Process . In brief, 
the process functions as follows; 

o Stimuli are furnished by NASA Centers and/or program offices, who 
cite a NEED together with a justification for such a need, and an 
accompanying GoS )'Worth analysis. 

o Reaction causes a Specifications Library check to be made, using 
an high speed computer and printer to perform the Investigation. 

The initial Cost/Worth analysis is verified within the CSCO, and 
the process of either gristing document updating or new document 
Generation begins. 

o Distribution of the drafted copies is undertaken by the library 
and dates are set for: 

o Response by all addressees, whose comments -and technical critique 
cause the preparation of: 

o Revisions This process is subject to: 

o Iteration for as many passes as are required to refine the document. 
As a general rule, iterations will be held to not more than 3 to 
minimize time -in-t he -loop, and allow: 

o Final Release of the specification or other document to users. 

° APP lica -tiori can then begin throughout MSA and the aerospace industry 
2.3 Detailed Work Flow: Based upon the foregoing logic, a detailed work-flow 

diagram. Figure 2.3.1. Central Specifications Control Office Work Flow , is 
include a. This figure, which is self explanatory in terms of CSCO operations, 
illustrates the options, specific task functions, work processes, and release 
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system. 

2, ^‘ CSCQ Task Priorities : Much has been written by LMSC and other aero- 

space companies concerning the specification proliferation, duplication, 
and obsolescence in the present system. In addition to these factors, al 1 
of which tend to drive program costs upward, as is shown in Figure 2 . 1 ; 
the present specification hierarchy is a mix of Federal, DoD, Tri-Service 
and NASA documents. The first task recommended for implementation by the 
CSCO is that of creating 1 some order from this chaos. In particular, this 
task will require screening the entire aggregation of documents currently 


existing; removing those which are outdated, or are duplications of other 
documents, and recoding/cross referencing the balance with a NASA coding/ 


numbering/titlmg system. The entire library, which will be stored in'micrO' 


fiche/microfilm form, should be coded and computer referenced. When the need . 
for a paper copy arises, this can be reproduced by a combination of the Ki- 
Speed printer for text, and an XT plotter such as the SC J;020 for graphical 
data. Each Center can be equipped with a CSCO interrogation facility which, 
by means of leased lines would permit a Center CSCO Rep to acquire a specific- 
ation copy in a rapid fashion. Either text, by hispeed printer, or art by 
Facs (facsimile transmission) could be available, or both, as needed. 

2 *£ The Center CSCO Representative : Such personnel, who would probably be ass- 
igned part-time only, would be specialists in one or more engineering discipline, 
of one or more of the assurance; disciplines. In addition to their expertise. 


they would require experience in the generation of technical specifications. 
The problems existing currently are that most personnel do not know what 
specifications exist, what these cover, their state of currency, and the work. 


confusion, and hence costs, arising from their implementation. Sue 
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applicable or not. 1/hat is worse, all of the sub-tier documents are invoked 
automatically, by dependency. The major duty of the CSCO Rep at each Center 
will be to interface with aerospace contractors, other NASA Centers and Prog- 
ram Offices to: 

o Advise what specifications/documents will be needed 
o Advise what exists currently, and what new items will be required 
o Advise of problems arising from imposition of current specifications/ 

, documentation requests upon the aerospace industry. 

o Process requests for CSCO to produce new specifications, update current 
issues, and expedite response time to such requests, 
o Coordinate CSCO policies & procedures with the Center to which he is 
assigned. 

o Provide cost data on specification implementation, also cost targets. 

3. Conclusions : An urgent need exists for NASA to undertake major revision of 
the specification hierarchy existing presently. This need has been highlighted 
in numerous studies undertaken by LM3C and other aerospace contractors over the 
past 3 years. While this EM can do no more than suggest the groundwork for a 
NASA Central Specifications & Documentation Control Office (CSDGO), it urges 
that NASA consider such an office to be a near-term priority. LMSC believes that 
considerable savings of the order cited in the opening paragraph of this EM 
may be achieved, merely by improvement of the specifications used to regulate 
program performance. Finally the office, as has been shown herein, can be used 
as an effective means for the control of all forms of program documentation j 
both NASA and contractor generated. The CSDCO can proive a means of realizing 
the intent of the slogan: "Less paper; more and better hardware for the NASA 
Program Dollar Expended." 


Lockheed Missiles & Space Company 
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SYSTEM ENGRG ^ 


1. Introduction 


i^April’ 1969 ) Quality Prograin Pr °^.sions for Space System Contractors was issued 

The document replaced NPC-200-2 dated April 1962, and was an attempt to update 
and streamline the quality provisions mandated by NASA for controlling the 
quality of spaceborne hardware . As such, the update and streamlining process 
was extended to include the reliability activity, the soldering activity, and 
the inspection activity, so that the complete NASA Handbook included most of 
the assurance support activities, with the exception of maintainability, 
safety and logistics. At present, the NHB series is structured as follows: 
o NHB 5300. ii(lA) which covers reliability , 
o NHB £300.1*(1B) which covers Quality Assurance . 

0 NHB 5300.1; (1C) which covers Inspection. 

2. Handbook Specifics 

The handbook, while organized as a separate document, is structured along the lines 
of the DoD document . MIL-Q-9858 A. As such, the handbook shares not only the strengths 
of the DoD document but also the disadvantages. In the most general sense the 
paramount problem with both of the documents is not that anything is missing from 
them, but rather that the degree to which activity prescribed must be accomplished 
is not specified. Secondarily, there are several inclusions descriptive of program 
activities, which do not properly fall within the purview of Quality control, 
but which appear to have been included for lack of a better place to treat them. 

2.1. Organization of the Handbook 


The handbook is organized into thirteen chapters as follows: 
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Chapter 1: Introduction. 

Chapter 2: Quality Program Management & Planning 
Chapter 3: Design and Development Controls 
Chapter h : Identification and Data Retrieval 

Chapter $ : Procurement Controls v 

Chapter 6: Fabrication Controls 
Chapter 7 : Inspections and Tests 

Chapter 8: Non Conforming Article and Material Control 
Chapter 9' Metrology Controls 

Chapter 10 : Stamp Controls ' 

Chapter 11: Handling* Storage, Preservation, Marking, Labelling, Packaging, 

Packing and Shipping. 

Chapter 12: Sampling Plans, Statistical Planning and Analysis 
Chapter 13 : Government Property Control 
Appendix A: Quality Program Cross-Reference Index 
Appendix 3: Glossary of Terms. . 

Each of these chapters is further sub-divided into several categories, most of 
which commence with the mandate: "The contractor shall..." What the contractor is 
required to do is quite clearly prescribed. How the contractor is to do it, or 
whay it should be done at all, is usually not stated, or if a statement IS made 
it is couched in vague, general terms. MIL-Q-9858A possessed similar deficiencies, 
to such an extent, that in an attempt to remedy them the DoD produced Handbook $0. 
This document purported to supply the "How" and the "Why", but in actuality provided 
more confusion than clarification. 


3 . Cost Driving Aspects of the Handbook 

3.1 NASA vs The Aerospace Industry in Program Planning 

The number of contractors who are capable of marshalling the resources required 

to bid for NASA space programs is relatively small. After almost 20 years in 

LOCKHEED MISSILES & SPACE COMPANY. INC. 
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designing and fabricating space vehicles, the individual contractor has amassed 
considerable expertise in the provision of a quality product, and the "ground- 
rules" by which such provision is accomplished are well established. Despite 
this condition, NHB 5300J 4 (1B) requires that for each, and every program, con- 
tractors prepare, and submit for approval, a quality assurance plan. This plan 
is to be readily identifiable with each cited requirement (which mandates its 
format ) , must cover all quality activity throughout the time period or phase 
authorized, must be updated periodically and re-submitted for approval, and must 
specify exactly how the contractor proposes to be responsive to the' requirements 
of NHB 5300. U(1B). In practice, what transpires is the production of a plan 
of the boilerplate variety. Each contractor submits the same kind of a plan, 
with statements tailored to match the paragraphs of NHB 5300.h(lB), for every 
NASA solicitation upon which the contractor bids. Relatively speaking, apart 
from the cost values involved in the programs, there is no variation program to 
program, in any of these plans; nor is there much variation contractor to contract- 
or. One of the requirements to be met is the provision of work-flow diagrams 
which show the quality audit points, and inspection points throughout the program. 
As may be imagined, on a large program such as the shuttle, the program plan 
becomes a sizable document. Program plans of this kind, which often are required 
to be submitted in preliminary form with the contractor's response to the NASA 
RF p , can run as high as $25000.00... As was stated earlier, the relatively few 
companies providing space hardware for NASA operate in similar fashions, each 
to the other. It would appear more cost effective for NASA to retain a SAMPLE 
plan for each contractor, and require additional program planning for space 

programs only if there are special problems envisioned, or novel methods to 
be used. 
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3*2 The NASA/Contractor "Paper Mil]. 11 

At the end of NHB 5300.1* (IB) there is an Appendix A, Listed in this document 
appendage are 1*0 document titles, each of which is cross referenced to one 
or more paragraphs of the NHB 5300.1* (IB), if the provisions of any given NASA 
contract are strictly enforced by a contracting officer, each of the documents 
listed in the Appendix must be prepared, updated, and maintained for the life 
of that contract, and in some cases for a protracted period thereafter. Obviously 
many of the documents exist from contract to contract, for example the Change 
Control Sysyem Document, which merely, notifies NASA that a change control system 
is in operation and is administered in some given manner. The significance of 
this appendix is that per contract every one of these documents must be prepared, 
reproduced, and transmitted to NASA should full compliance be mandated. In actual- 
ity those required are listed in the CIRL, and this list includes most of them. 

It has not proved possible to estimate the expenditures per contract involved 
with compliance to this documents appendix, yet it is apparent that such costs 
may be appreciable. If the list is to be retained at all, it should be scrutin- 
ized carefully by NASA to determine which of the documents is really necessaiy, 
the criterion for document retention being what impact it has, or may have, 
upon the final quality of space destined hardware. 

3*3 Government Property Control 

Chapter 13 of NHB 5300.]*(1B) deals with the control of Government Furnished 
re sponsibilitie s 

Property. The xsspxs±ki±ii±ES of the contractor are mandated with respect to 
the accountability, maintenance, shipment,; periodic inspection accorded to 
Government Furnished Equipment. In addition, the contractor is given instruct- 
ions on the handling, reporting, segregation and disposition of any GFE found 
to be defective. 
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Particularly, the contractor Is instructed to proceed in accordance with 
Government Procedures, when confronted with unsuitable OPE. „hat procedures 
are involved are not specified in this handbook, but in fact Fed-Std-717 
Control of Government Sporty is involved. WSC has not been able to detente 
what portion of Quality assurance funds are emended on this activity customarily, 
nor are there well defined records of what OR NASA furnishes to what particular 
contract or program readily available. On a large program Such as Apollo however 
where the involvement of GFP/GFE was considerable, control measures for such 
property/equipment cost the program several million dollars. W3C believes that 
Government Property control should not be made a part of Quality Assurance Pro- 
visions. Rather the property should be handled and processed like any other bonded 
inventory items. Since Government Property is usually facilities and/or large 
test equipment items, the responsibility of handling such, should be vested in 
general inventory control and accountability. Supervision of this operation 
should be undertaken by Program A^nistration rather than Quality Assurance, 
and the particulars of what the Covenant requires should be covered in 
the general provisions of NASA contracts. 

3 ,k Non Conforming Article & Material n^^i. mt 

in C hapter of the Quality ^ drlVlng 

iniected into ^vxsaons document is the delay that can be 

Bo^d The Hat IT “ “ “ ° f t!>e “ *“>* <* inertia of the 

Board . The Material Review process requires that an initial assessment of non 

ed “aLT* 1 ^ “ aS ™ ° f the “—tor involv- ' 

Whose ■ H these personnel must then be forward to the on-site board 

whose members review the situation and make decision as to disposition of the 
discrepant materie^tems . !n all cases where the decision 

\° V — °° ard recotnmne nations must be forwarded to the NASA 
contracting officer (COR) for concurrpnno ** xm 

as ones-of-a-kind fn J • C ° nCUrrencs * ^quently costly items are purchased 

fabrication A d — + ° n ° ^ haS — provided > ^her by buy action or 

laoncation. A decision to repair at the vendors' nlanf- ^ - , 

tractors' facilities »,=+ * ‘ r a re-work at the con- 

to slmolif y 0311 f0r SChedUla Recommendations 

o sajnplixy ac’tivx'ty in rnat/eri v. n ,r; 

, m material review will be treated elsewhere in this EM, 
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3.5 Architecting the "Empires ": One of the difficulties of a document such as 
NHB 5300. U( IB), lies in its lack of specific direction. Many of the statements 
of requirements commence with the phrase, "The Contractor shall Provide.... 
Contractors have a vested interest in providing that which the customer requires, 
since failure to do so could result in an adjudication of non-compliance with 
customer requirements. The problems arises from the fact that many of the tasks 
that the contractor is directed to perform in the Quality Provisions document, 
are not quality tasks. Each of these cases will be treated in the detailed anal- 
ysis of the provisions of the handbook which follows this description of its cost 
driving aspects, but one or two examples are given to illustrate the difficulties. 
In chapter 7 dealing with Inspections and Tests, paragraph 1B702 deals with the 
preparation and use of test specifications for each test to be performed. Since 
the paragraph is contained within the quality provisions document, contractors' 
quality organizations have construed such a direction to apply directly to them. 
The outcome has been the institution of organizational subdivisions known as 
"Quality Test Operations" or other similar titles. In actuality, what NASA really 
intended was that Design, and Systems Engineering determine and specify what test 
should be performed, at what levels, Test Operations personnel should specify 
equipment requirements, make such equipment available on-schedule, and perform 
the tests, with program management exercising supervisory control of the entire 
operation. The role of the assurnace disciplines in the test function should be 

only those necessary to assure that: 

(a) The tests have been performed to the number, and duration of 

tests specified. 

(b) The test criteria have been met, or not. 

(c) Discrepancies are identified and handled. 

(d) All test activity has been incorporated into the program perm- 
enant record. 

Such procedures, when implemented would assure that costly duplications of the 
test function would not occur. 

Similarly in Paragraph 13603 which deals with Process Controls, such controls 
are intended to be specified and undertaken by Process Engineering. The only QA 
role should be that of assurance that this activity has been carried out, and 
that the processes are under control. 
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Again, specifics will be given in the detailed analysis of the specification, 
but an example of the cost driving aspects of costs occasioned by the quality 
document be so general as to allow arrogation of many responsbilities to the 
quality organization, not properly theirs ; may be found in the fact that on 
several programs undertaken by NASA, QA costs have run as high as h3% of the 
manufacturing costs J 

ii.O. Digest of Contents, NH3 5>300»Ix(lB) : As has been listed earlier in this EM, 
the Quality Provisions Handbook contains thirteen chapters. For ease of under- 
standing the provisions of the handbook, and to facilitate reading this EM without 
the necessity of recourse to a copy of the handbook; the following digest is 
provided: 

U.l Chapter 1, Introduction : This chapter, which has been assigned alpha numeric 
designators 1B100 thru lBlOli introduces the theme of the handbook. The chapter 
furnishes a general preamble, the relation of quality assurance to other contract 
requirments, sets forth actions and prerogatives of the government, details 
documents required by the quality program cross referencing these to applicable 
paragraphs of the handbook, and furnishes a glossary of Quality Assurance terms 
used. Of particular interest is the paragraph dealing with actions of the govern- 
ment, and its prerogatives. This informs the contractor that all aspects of the 
quality program he institutes are subject to governmental examination, by either 
the NASA, or other surveillance agency of the government appointed by NASA as 
its representative. 

14.2. Quality Program Management and Planning : The chapter covers designators 
1B200 thru lB206treating Generalities, Organization, Training, Quality Information 
required. Quality Status Reporting, Quality Program Audits, and the Quality Prog- 
ram Plan. The thrust of the chapter is to detail to the contractor how must be 
managed the quality program he is required to institute. The significant paragraph 
is the last one, which deals with the quality plan which is to be prepared for 
every program. This plan is required to cover all other requirements set forth 

in the handbook. 

14.3. Design and Development Controls : Only three subjects are covered in this chap 
ter. 1B300 thru 1B302 set forth the quality involvement in technical docUmentaton, 
the support Quality Assurance is to furnish to Design Reviews, and the role QA 

is to play in change control. Each paragraph is generalized to the point where 
it is difficult to determine what -QA really is required to provide. 
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k»k* Identification and Data Retrieval : The contents of this chapter are 
designated IBI 4 OO thru lB!+05. The subject matter covered i? a General Preamble 
which explains the import of the chapter. Methods of Inde;: uification by Date 
Codes, Lot Numbers, Serial Numbers, and other methods permissible, Documentat- 
ion requirements in terms of identifications, and part numbers, control of ident- 
ification numbers, the provision of lists on which all items contractor, and 
supplier- designed, shall be identified by part and type number, and the methods 
of verification of parts/materials versus identification lists, versus pertinent 
program records. 

1|.5« Procurement Controls : NASA attaches considerable importance to the control 

of contractor procured items. This chapter contains eleven (11) sub-chapters, 
which detail for the contractor thru sections 1B500-1B510 inclusive, his respon- 
sibility for all purchased articles, materials, and services, Quality Assurance 
involvement in the .selection of procurement sources, and the criteria covering 
such involvement, the review of supplier quality programs for adequacy and com- 
pliance with the requirements of the NRB Handbook, all of which requirements, are 
listed in some detail, the assignment of contract quality personnel at suppliers' 
plants and facilities, and details of how this must be done, a statement of the 
intent of Government Source Inspection, which, as, and when conducted, in no way 
relieves the contractor of his quality responsibilities, the receiving inspection 
system required of the contractor, and elaboration of that theme, the receiving 
records required, and the manner in which they are to be kept, the system for 
rating suppliers in terms of the quality of their product, the requirement for 
post-award surveys, and the criteria for their conduct, the need for co-ordin- 
ation of contractor-supplier inspections and tests, the requirement for feed- 
back, contractor to supplier, of information on non conformances to facilitate 
timely corrective action, which is the final paragraph, IBjlO. 
lj. 6 . Fabrication Controls : This chapter contains $ subchapters; 1B600 thru IB 
60 J 4 . These cover the controls required throughout all phases of hardware fabric- 
ation, the controls required to assure that only conforming articles and mater- 
ials are released for use; such controls being inclusive of critical life articles 
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and items; the control of cleanliness throughout allphases of the fabrication, 
assembly, test, storage and shipping operations, the controls to be provided 
for those articles whose conformance cannot be dete mined by inspection alone, 
these being particularly oriented toward such processes as plating, chemical 
processes, et al., and finally the criteria for the joint selection by NASA and 
the contractors, of samples, and/or visual aids showing acceptable workmanship. 

11. 7 . Inspections and Tests ; There are 8 sub-sections in this chapter, numbered 
1B700-1B707. They include a general statement concerning the requirement to plan 
and conduct an inspection and test program, the criteria against which such a 
program is to be planned and conducted, the requirement to use test specificat- 
ions for each item to be tested and each test to be performed, and details of 
the contents of such specifications; the requirement for the contractor to 
prepare written procedures for tests and inspections to be conducted, with 13 
sub-criteria listed, a statement that separate end-item inspection and test pro- 
cedures shall be prepared and used together with applicable criteria for these; 
a considerable treatise on the methods and requirements considered mandatory for 
inspection and test performance, with stress on the control of Qualification 
Test Articles, the conduct of Requalification Testing, edicts concerning Qualif- 
ication Based on Similarity, and the criteria governing inspection and test of 
end items. This latter sub-section is cross referenced to chapter 8 with respect 
to the manner of dealing with non-conformances discovered during and after test 
and prior to shipping, and includes requirements statements concerning re-test 
activity, and the preparation of End-Item Inspection and Test Reports. In addit- 
ion to all of the foregoing, the contractor is required to generate Inspection 
and Test Records and data concerning all test activity undertaken, and finally, 
a list of Contractor Quality Assurance Actions to be taken before testing com- 
mences /is ; furnished. . • 

1 4 . 8 . Hon-Conforming Article and Material Control : This chapter has 7 sub-sections. 
These, designated 1B800 thru 1B806, cover the definition of non-conformance, 
documentation required of the contractor in his program to control non-conform- 
ing materials and items, the prescribed remedial and preventive action the con- 
tractor and his suppliers are expected to take, the requirement for initial 
reviews & dispositions to be made by contractor quality personnel, together with 
details thereof, a description of the Material Review Board together with its 
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membership, functions, and authority, a statement of requirements for 
Written Request For Approval by the NASA Contracting Officer to be generated 
whenever a contractor recommends a disposition to repair, or use items' as-is, 
and finally, permission with NASA approval for the contractor to delegate MRB 
responsibility to suppliers, if the contractor so elects. 

i|»9. Ifetrology Controls : This chapter, which has 8 subsections designated from 
1B900-1B907, deals with the documented system which the contractor is to use 
for control of the measurement processes evidential of quality performance. 

The subsections deal with Acceptance of all standards and measurement equipment 
prior to use by an inspection process, the means of evaluation to ensure the 
validity and compatibility of measurement standards to the hardware they are to 
measure, the processes by which measurements are to be made, the calibration 
process for the standards, together with its details of traceability, handling, 

• I • . ' " 

storage, and transportation of the standards, their identification and labelling. 

‘ j 

calibrations and their intervals, the recall system for re calibration, and the 
calbration records requirements. In addition to all of the forgoing, the require- 
ment that environmental characteristics to be provided shall be compatible with 
both the article to be measured and the measuring equipment; and finally, the 
remedial activity prescribed forboth measurement equipment and articles under 
measurement which exhibit non-conformance. 

I;. 10. Stamp Controls : Chapter 10 has only two subsections, 1B1000 and 1B1001. 

The first covers a six point stamp control program that the contractor is requir- 
ed to implement and maintain, and the written procedures to be prepared to cover 
it. The second sub-section contains a restriction stating that contractors' 
stamps shall not contain the designation' NASA. ' 

If. 11* Handling, Storage, Preservation, Harking, Labelling, Packaging, and Shipping 
This chapter covers what controls the contractor is required to provide in the 
areas denoted by the titles given in the chapter. The coverage is stated in 3 
sub-sections, 1B1100 thru 1 B1102 . Respectively these deal with requirements 
for handling and storage of hardware, Preservation Marking and Labelling, Pack- 
aging and Packing of hardware to obviate deterioration, and control of Shipping 
of articles by means of a 7 point prescription, covering both hardware and its 
associated documentation package which is to be shipped with the hardware. 
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U.12 Sampling Plans, Statistical Planning and Analysis : This chapter has two 
subsections only, 1B1200 & 1B1201. The first deals with the conditions under • 
which NASA will permit the use of statistical sampling plans and techniques, 
and the requirement that all such usage requires approval by NASA or its 
elected representative government agency. The second section deals with the use 
of statistical quality control methods and associated charts, and the situations 
and conditions under which such methods are permitted to be employed. 

U»13. Government Property Control : This final chapter in two subsections, 

1B1300 & 1B1301, defines the contractors responsibility for the care of any/all 
property furnished by the Government for use with a given program or programs. 

A 5 point action list is furnished to cover both equipment, and records to 
be maintained with respect to said equipment, as well as a requirement for fun- 
ctional testing to assure that the equipment is in proper order. The second 
sub-section requires the contractor to segregate, identify, and notify the 
Government supplying agency of any damaged, malfunctioning, or non-conforming 
Government property. None is to be reworked without authorization by the supply- 
ing agency. 

5 .0 Detailed Analysis and Recommendations for Changes : The information contain- 
ed in this portion of the EM is not intended to be a complete re-write of NHB 5300 
,U(1B). Rather the intention of this analysis is to treat the document chapter - 
by-chapter, and paragraph-by-paragraph, in sufficient detail to illustrate 
either the inadequacy of the document in its present form, or the cost driving 
contributors, or both. The level of detail given, and the recommendations for 
changes believed to be cost effective if adopted, are furnished in such a manner 
as to permit a rapid re-write of the handbook in a more specific form, should 
NASA elect to implement such a process. This outcome dictated the choice of 
presentation format for the material. The format presents the left hand side of 
each page by chapter and subsection as it Is Now . The right hand side of each 
page shows the subject matter as it is TO BE , assuming that recommended changes 
are adopted. 

5.1. Recommended Combination : Quality Assurance as a discipline embodies both 

Quality Engineering and Inspection Activities. As an over-simplification, 
it is the function of Quality Engineering to determine the most effective meth- 
ods by which the quality of hardware/software may be assessed, and to promulgate 
standards of workmanship etc. Inspection functions seek to determine that the 
hardware meets the quality standard specified for insurance of proper operation/ 
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end-use function. Methods employed to make such determinations range from the 
simple, such as go no-go gages, and visual checks of form fit and function, 
to the complex where optical comparators, electronic metrological devices, 
chexcal processes, and other aids are employed to determine the conformance of 
hardware/software to the design criteria against which they have been produced. 
Since the two disciplines go "hand-in-hand" it is difficult to divorce one from 
the other. IMSC recommends that Inspection and Quality Provisions be combined 
ultimately into one integrated document. Currently, Inspection is treated as 
a separate entity forming another part of the NHB. 51^00 series, specifically, 

NHB 5300.1* (1C). 

5«1-1* Example of Combined Document Attempt : In an attempt to provide for the 
assurance disciplines for the Shuttle Era, NASA has issued a document, 

NHB 5300 .^(lD-l) Safety, Reliability, Maintainability, and Quality Provisions 
for the Space Shuttle Program. Essentially, this document combines under one 
cover, MB 1700. 1, MB 5300. 1*(1A) MIL-STD-U70, and NHB 5300. 1| (IB). Each still 
stands as a separate autonomous document, and while there is additional material 
incorporated into each, to allow such considerations as "refurbishability", 
turnaround time, hazard analyses, etcj LMSC does not consider that the document 
has been prepared with Low Cost as the prime criterion. Much of the content is 
word -for -woi'd repetition of the current unmanned program documents. This docum- 
ent is quite recent, date of issue being August I 97 I 4 . 

5.2. Analysis Tabulation: 


Chapter "Is Now" 


Chapter "To Be" 


Chapter 1 . Introduction 

1B100 General ; Statement coveij 

ing requirement necessary to 
provide effective operation oij 
a quality program for NASA 
aeronautical & space programs 
systems, subsystems, & relat- 
ed services. 

1B101 Relation to Other Con- 
tract Requirements ; Covers 
overlapping of other contract 
rqmts, such as rel, safety & 


Chapter 1: Introduction... No change, 
1B100 General: Stet. 
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Chapter "Is Now" 

Chapter "To Be" 

1B101 Continued: 

1B101 Stet. 

and test, shall not result in 
duplication of contractor 
effort. 

1B102 Actions & Prerogatives 

. 1B102. This subsection contains items 1,2 

of the Govt. 

Covers the fact that all 
contractor effort is subject 
to NASA evaluation, review, 
audit etc. by NASA office 
procuring or Govt elected 
alternate . 

item 1 deals with the subject matter as 
shown at left. Item 2 covers the resident or 
itinerant basis on which Govt Reps may be 
assigned. Item 3 covers facilities & services 
contractor is to provide to Gov't Reps. 

All Stet. 

• Item 3 should be amended to state uneqivocally 
what is expected, but this amendment should be 
elsewhere in the contract preamble, rather 
than change the statement in this handbook. 

13103 Quality Program Docmts: 

1B103 . NASA states in CIRL/CDRL which 
documents it requires in the Approval, 
Review, and Information categories. 

' Recommend STET for the paragraph, but ADD 

P.eference is made to Appcix A. 
This Lists [|0 program doc'mts 
all cross ref'd to the KDBK 
by sub-para number. Any/all 
are subject to (a) Approval 

(b) Review, (c) Information. 
Explanation of these three 
categories is given. 

Item 2 requires contractors 

"Documents requiring NASA approval will be held 
to a minimum to include only.... per CDRL." 

"Documents subject to NASA review shall be held 
to a minimum and include only those agreed 
essential to the successful prosecution of 
the quality program; such agreement to take 
place during contractual negotiations. 

"Documents required by NASA for information 
will be held in the contractor's files and 
furnished to NASA only on request, and In 
the minimum number of copies possible. 

Item 2: Recommend deletion, staement is 
superfluous, contractors are aware of need. 

to generate documents cited 
and hold them available to 
NASA. 

IBlOlu Glossary of Terms: 

IBIOU: Stet. 

Defines terms used in this 
publication. 


13200 Quality Program Mgmt & 


Planning 

Outlines a 5 point approach 
contractor is to take to 
assure quality. Includes 
recognition of necessity 
for all aspects of Quality 
program, rqmts determined & 
met, designs include quality, 
Detection of deficiencies 

— — i 


REPRODUCIBILITY OF THE 
ORIGINAL PAGE IS POOR 
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Chapter "Is Now" 


Chapter "To 3e" 


1B2Q0 Gen'l (continued) 


Provision of corrective 
action. 


TB201 Organization. Cites the 


mandate for a Quality Org'n 
with one person responsible 
with direct access to higher 
rngmt . 


. 1B202 Training : Sets forth a 
'll point training program, in 
which Workmanship is to be 
assessed, operations are to 
be assessed in terms of care 
and Safety , personnel are 
to be certified as proficient 
in their respective operat'ns 
recert'fd periodically, and 
training records kept. 

1B203 Quality Information: 


Covers contractor's respon- 
sibility to furnish qual info 
thru-out his org. & to supp- 
liers, on all aspects of prog 

13201 Quality Status Rptg 

Outlines rptg on 1 point 
basis. Personnel changes(key) 
Problems & Solutions, perf- 
ormance, & supplier perf'mce. 

13205? Quality Program Audits 


Covers contractors respon- 
sibility to ’heck on quality 
program, all aspects to deter 
effectiveness. Audits to be 
random, outcomes reported to 
higher contractor mgmt. 

1B206 Quality Program Plan: 


Mandates a Quality Program 
Plan 


13200 : The statements set forth are innocuous. 
For past 1 $ years every aerospace contractor 
has maintained such a Quality program routinely] 
Statement does not drive costs, but neither 
does it add anything to the Hdbk. Recommend 
deletion, 

1B201 Statement is innocuous. Recommend add- 
itional statement, to show one person for each 
program , to manage Quality functions. Dual 
reporting authority, (a) Program Mgr in direct 
line, advisory to overall contractor Quality 
management . 

1B2Q2 All major aerospace contractors have on- 
going training programs of type specified. 
Rather than inform NASA of fact in every RFP 
response, and show some costs for training in 
each cost proposal, recommend: Amend statement 
to read: Contractor shall maintain a quality 
training program which NASA will review 
periodically for adequacy. 


1B203 Stet 


13201; Amend statement to include. Wherevei 


possible, contractors shall report these data 
as part of other program reporting to minimize 
paperwork, and cost of reporting. 


13205 .' Contractors perform such functions rout- 


inely, as a matter of business practice. 

Such a function does not need to be specified. 
Delete the paragraph. 
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13206 Continued ; 

itTS submitted with either WBUm - 

RFP response or as per contract rev i ews s 

, _ 1 Hiiol ‘ . ■ • 


i'-i X - * _ 

States plan to cover all Qual 
[tasks, separate plans for 
each remote site, updated 
j periodically. Covers by U 
points exact contents of plan 
including work-flow charts. 


Chapter "To 3e" 

13206 Every aerospace contractor doing 
a s's "with NASA has responded to this rqmt for 
years, for PROGRAM UNDERTAKEN . 

'LMSC reviews show that each contractor's 
plan and revisions, closely resembles that of 
his competition. Apart from the purely paper- 
work oriented costs, review time is c °f t 
driving for all these plans. At current le /els 
of capability contractors do.not need to 
told how to plan quality activity. 

-Recommend: Section be amended to require a 


standard quality plan , submitted °nce yearly 

• 4- attd r* a DT?02ic 


Stand ara quaiiuy ,n W nwan:s 

by each major contactor, to cover all programs 

undertaken by and for NASA. Program P^ arit 
ies onlv need 1 be costed and submitted with 
the RFp“ response, and these only when signif- 
icant departures are needed in current qua j 
technology and practices. The plan should 
include Inspection activity in the _ standard^ 
publication, and only significant departures 
from norms should require work-flow charts 
to be prepared new for each program. Off -site 
peculiarities, and remote site peculiarities 

reiuira update rather than new planej 

each program. 


Chapter 3 . Design £ Develop - Chapter^, 
iment Controls. Sets forth: , ., n 

1B300 Technical doc'mts rq m ts 1B300: Stated ^“y^S^cs^ erit- 

1B301 Qual support to Des.Rev docent al MJJ* ^f not or iented to 
1B302 Change Control ena. 


ud.« ilicoc — — 

functions an d responsibilities . 

- 1 * 1 A e civ 


0A runc'Oiuuo <xii ^ i. ^ _ 

Part 2: Requires tnat Quality Assurance per - 
onnel review all program documentation, for 
determination of all deficiencies, clarity ox 
statement, unambiguity etc. This also is nou 
a. Quality function primarily. 

Recommend; QA responsibility to check document- 
s' be limited to only those directly relatec to 
Quality functions, i.e. Quality should not need| 
to check drawings, engrg checkers do this, t 
is not a QA function to assign, determine, or 
check characteristic tolerances, only that _ 
hardware produced meets the tolerances assigned 
and checked as valid by engineering. 

13301 : QA participation in design reviews 
should be limited to Inspect ability only. 
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Chapter "Is 
1B301 Continued. 


Now" 


Chapter "to Be" 

The determination of producibility, and 
repeatability are not Quality Assurance 
functions, but rather those of Process Engrg 


1B302 Change Control : 

Outlines the contractor's 
responsibility to control all 
documents and changes thereto 
affecting the Quality Program 
Requires the change control 
system to be documented. 


13302. Recommend amending this subsection . 
Delete the requirement to document the change i 
control & release system, as this would form 
a part of the standard Qual.Plan. i 

Part . 2 . Ef festivity ; Recommend deletion of this! 
clause. Function is a compound one involving : 
program management for scheduling changes, 
manufacturing for effecting the changes, as 
well as configuration management to determine | 
they are carried out. j 


Chapter It. Identification & 
Data Retrieval: 


Comprises : 

131+00 General : This requires 
contractors to develop & 
maintain an identification & 
data retrieval system to 
provide I.D. for operating 
fab, processing, procurement, 
inspection & test records. 
Means of tracing items & mat' 
These to be developed along 
with logistics & config mgmt. 
Common l/D numbers for all 
systems. 

lBijOl l/D methods . Covers 
3 part methodology by date 
code, lot number, serial no., 
and other permissible l/D 
13lt0 2 Documentati on : Requires 
system to be documented. 

1BL03. l/D Control: Covers 
traceability of items. 


lBitQO . This is not a Quality Function. Should 
be deleted. (Note this, and the ensuing other 
subparagraphs listed refer to items which on 
programs such as Atmosphere Explorer have been 
handled by Configuration Mgmt. This program 
combined Rel, Qual, & Config Mgmt functions 
and plans successfully, for approx half the 
‘.usual assurance disciplies costs.) 


lBltQl . Recommend transfer of responsibility to 
Config Mgmt. deletion from Quality HDBK. 

1BI|Q2 Recommend deletion. Transfer to Config. 
Mgmt . 

lBli03 • Recommend transfer to Config. Mgmt. 


IBliOlt l/D List: Covers listing ;lB)i Pit : Recommend transfer to' Config Mgmt. 
of all articles by number at 
onset of design activity. 

lBhO^. Records Retrieval.. 13lj05>. Recommend transfer to Config Mgmt. 
Requires that records be ref- 
erenced to articles & mat * Is 
and records be retrievable 
readilv. 
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Chapter "Is Now" 


Chapter "To Be" 


Chapter 3 ■ Procurement Cntrls 


Covers : 


1B$01 General: Contractor 
responsible: for quality of 
all purchase d ( c ont rac t or ) 
articles, mat 'Is, & services. 

1B502 Procurement Documents: 


Subparagraph covers review 
of documents as item 1 . 

Data documents must contain 
as item 2 . Item 2 covers 
sub-items a,b,& c for 
Supplier Qual Programs . 

Basic -lech Rqmts . 

Detailed Qual Rqmts . This 
latter item 1 c > covers 13 
subjects: Changes, Purchased 
Raw Mat'l, Raw Mat'l Used in 
Purchased Articles, Preserv'iji 
Packaging, Packing & Shippin| 
Age control, & Life Limited 
Products, I/D & Data Retr'vl 
Inspection & Test Charset' tc^ 
Insp'n & Test Records, 
Re-Subm'n of Non conform' g 
Items or Mat 'Is, Contractor 
QA at Source, Gov't Source 
Insp'n, Procurements other 
than those requiring GSE, 
and Equipment Records. 


163 03 • Contractor QA Person - 
nel at Source : Permits the 
contractor to assign QA pers 
-onnel at suppliers/ subcontra) 
-ctors. If so assigned, 
personnel are to be described 
in terms of duties to be 
performed, to Gov't QA Rep. 

6 criteria for assignment are 
listed, one or more of which 
must exist. 

IBguU . Gov 1 1 Source Insoectioil 


1B500 

1B501 • Stet . for both the General statement, 

nation, tBg l»lllao?i n SP??SS8J4gnr ral - 
program sources. 

1B$02 Stet for this entire subparagraph in 
all 13 sections. Quality Review is to assure 
the inclusion of these provisions in procure- 
ment documents. 


16303. stet ♦ But recommend addition of the 


statement, "To the extend feasible". The costly 
nature of such personnel assignments should 
carry a limitation to programs of some given 
dollar level. For programs at lesser levels, 
ININERANT QUALITY ENGINESRS/lNSPECTORS should 


be specified as an alternate approach. These 
can serve several contracts. Note: this is a 
permitted U.S.Mavy practice. 


Covers fact that Gov't Source 
inspection does not absolve 
contractor from resnosibilitvj 


1330^. Recommend Re -word to make G.3.I. 
sufficient where done. Two costly inspections 
are unnecessary. 
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Chapter "Is Now" 


13505 Receiving Inspection 
System 

Mandates that the contractor 
shall maintain a receiving 
inspection system. Sets forth 
12 assurances that the system 
must provide. These are ex- 
haustively specified. 


Chapter "To 3e" 


IB5>06 Receiving Records . 
Covers rqmts to keep records 
of all rcvg insp'n activity, 
and states minimum contents 
of such records. 

13507 . Supplier Rating System 


Requires contractor's to rate 
suppliers based on rcvg insp 
and test results. Outcomes to 
verify procurement sources. 

1B$08 Post Award Survey of 


Supplier Operations : 


Specifies a three parameter 
survey of suppliers to be 
done by a contractor post- 
award. Covers, complexity 
criticality of items, supplie: 
quality record, fab/test cap- 
ability etc. 

15509. Coordination of Con- 


15505 ■ Stet for the entire subparagraph. 

There are two alternatives to this recommend- 
ation: 

0 If NASA does not combine MHB 5300.1|(1C) 
which covers inspection into a re-write 
of the Quality Document, delete this 
entire paragraph and insert it into the 
(1C) Inspection Document. 

0 If NASA does elect to combine the two 
documents, this paragraph should be 
amended to denote which of the 12 assuranc 
-es are Quality Engineering oriented, and 
which Inspection services. Currently, 
there is a mix. For instance, item 8 
requires that chemical analyses be 
performed as required by specs or dwgs 
on test speciments. Hourly rated inspector 
personnel can only testify that such has 
been done. Engineering Test carries out 
the analyses, and Quality Engineers verify 
that the results meet performance rqmts. 

1B506 . Either STET 


as with°previoul ^Incorporate into (1C) 


15507 . STET. Quality Engineers actually per- 


this rating service. 


I 1 -.'.-’ . ...... . ; 

'15508. Recommend amend to read : "Contractor to 


penorra on first NASA program supplier is used 
only". Survey is costly, and after qualifying 
supplier .should be maintain on qualified. suppl- 
ier list unless cause arises to change his stat 
!us . 


tractor-Supplier Inspections 


and Tests. 
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Chapter "Is Now" 


13^09 Continued : Covers 
requirement for compatibility] 
of inspections/tests contract 
or/supplier. Contractor gives 
assistance to this end as 
rq'd. 

13510. Nonconformance Inform - 
ation Feedback. Concerns feed- 


back of non conformance info 
to suppliers, and assurance 0 
remedial action * 


Chapter "To Be" 


1B509. Re-write to show that Test Organization 


or Inspection services furnish training aids 
and technical assistance, NOT Quality Org. 
QA.has a surviellance role only. 


1B5 10 : Re-write to show that Quality Engrg 


Chapter 6. Fabrication Control 


handles feed back loop only. Diagnosis of 
Nonconformance is effected by Engineering. 
Quality Engrg handles corrective action follow 
up and re-certification, and Inspection handles] 
acceptance on next incoming shipment of re- 
worked items. 

Chapter 6. 33600, 601, 602, 603, 60q. 


Includes : 

1B6Q0 fabrication Operations 


IB'601 Article & Mat'l Control 


13602 Cleanliness Cont rol 
1B603 Process Control 


IB6OU Workmanship Standards 


Chapter. 7. Inspections & Tesi[ 


13700 Generah. Requires con- 
tractor to plan and conduct 
an Inspect' n & Test Program, 


This chapter illustrates the general nature 
of the handbook e Provision of the controls 
called out in this section are the respons- 
ibily of manufacturing, process, materials 
engineering support, materials engineering, 
and manufacturing planning organizations of 
a contractor, and/or suppliers. For instance 5 
the quality/lnspe ction role in cleanliness 
control is one of surveillance to assure the 
Federal Standards regulating cleanliness are 
met . In the case of para 1B60U, the Govt & 
contractor's quality personnel must agree 
upon the adequacy of the standard. The prep- 
aration of the standard sample or visual aid, 
and the process by which it is produced must 
be effected by engineering, process engineer- 
ing, and manufacturing. 

Recommend : (a) Delete entire chapter, incorp- 
orating data elsewhere in other applicable con| 
trol documents. 

(b) Re-write to show specific QA 
and Inspections roles, functions, and respon- 
sibilities. In the past QA has assumed the 
function of effecting the controls described 
rather than assurance that the functions have 
been provided by organizations qualified for 
such purpose. 


13700 . All contractors are aware of this need 
Recommend: Statements be deleted. 
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Chapter 'is Now' 


1B701 Inspection & Test 
Planning : Sets forth the aimsl 
and intents of the Inspect- 
ion- and Testing Operations 
to be provided by the contra-] 
ctor, & his suppliers. 


Chapter "To Be" 


1B701: Again, the statements are general in 
nature. They state what' is required of the 
contractor, not what is required as specific 
activity by his QA personnel/organization. 

This clause has three actual functions involved 

(1) Inspection Services. Provides Inspection 

(2) Test Operations. Plans, Executes Tests 

(3) Design Engineering: Design sets test limits 
The QA function is again one of surveillance to 
determine compliance only, NOT as has been the 


1B702 . Test Specifications : 
Covers the provision, and 
preparation of test specs. 
States what specs are to incl 
ude . 


1B703 • Inspection and Test 
Procedures. ~ " 


Covers the provision of 
suitable procedures for 
regulation of these operation 
and specific instruction of 
operating personnel assigned. 
Give a 13 point outline of 
content required. 


1B7Q^. End- Item Inspection & 
Test Specifications .'i Proced- 


ures. 


Requires provision of separ- 
ate Test & Inspection proedrs 
for end-items. These are to 
simulate end-use, without 
damage to end -item, and prov- 
ide a measure of quality. 
Degree, duration, & number of 
tests, to demonstrate contracij 


case, effectors, 


Recommend : Delete Paragraph/ Re-write to state 
Quality finite functions . 

1B702. Covers the entire gamut of rqmts. 


Includes Environments, safety, reliability, 

ing and analysis, re-test, and test results 
reporting. Role of Quality is only to assure 
that specs generated include such variables. 
Recommend : Re-word to show specific quality 


responsibility only, 

13703 . The inference can be, and in the past 
has been made, that a contractor's quality 
organization is to carry out the activity 
described as necessary in provision of these 
jO procedures. Of the 13 points covered, only ' 
item 13 Non-conformances is of direct concern 
to QA. The others concern Test Ops & Planning, 
Engineering, Safety Engineering, and Inspect- 
ion Services. The QA function is to assure 
procedures are available and. contents are 
as prescribed. 

Recommend : Deletion, and relocation of rqmts 


in test, ispection, & safety documents; or 
re-write to show specific QA responsibility. 
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Chapter "is Wow" 


Chapter "To Be" 


137 OU Continued : 

... compliance, and workman- 
ship merit 

1 1370$ Test & Inspection 
Performance : 

Covers.: 


1. Inspections & Tests . 

To be performed on procured 
and Fabricated hdwre prior to 
installation into next higher 
1 level. 

a. Control of Articles 

b. Control of Inspection 
& Test Environments & 
Equipment . 

c. Criteria for Inspection & 


2 . 


3. 


Re-Test : 6 criteria are 
listed. : 

Qualification Test Artic - 
les . 

Lists a,b,c & d controls 
Requalification Testing 

Contractor must state 
need for it, and gain 
Contracting Officer appro 
val prior to conduct. 

J 4 . Qualification by Similar - 
ity . States conditions in 
which NASA will permit it 

3. ISid-Item Inso'ns & Tests 


Contractor to effect such 
to approved specs. Handle 
non-conformances per Chap 

3 . • 

End Item Insp & Retest 
Requries Govt QA approval 
prior to conduct. 


6 . 


7. End item Inso &. Test REFT 


Covers format & content 
of report rq'd. 

1B706 . Inspection & Test 
Records & Data 


IB 7 OI 4 : Re c omme n d : De le t i on of this subsection. 
Quality has only surveillance role 
in this activity . 

1B7Q3 . All of the requirements & provisions set] 


forth in this complete paragraph, are general 
instructions only, and refer to what is requir-j 
ed in terms of specific tasks and reporting by 
the executing organizations involved, in this 
set of cases, Inspection Services, and Test 
Operations. In some areas, the chapter subsect-j 
ion merges into Chapter 8 as concerns the act- 
-ions of the Material d e view Bpard(MRB) in 
the disposition of non-conformances. The only 
function for QA throughout is Surveillance . 

This may be sub-divided into two parts: 

0 Surveillance, Test Operations & Inspection 
to assure actions are executed, 
o Surveillance, Inspection; advisory to 
assist in the interpretat 'n of critieria 
for inspection. 

Recommend : (a) Deletion of paragraph. 

(b) Re-distribution as applicable 
of these test and inspection requirements into 
other Test & Requirements for inspection doc- 
uments; i.e. Mil-Std -8l6, NHB 3300.1j(lC). 
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Chapter "Is Now" 


Chapter "To Be" 


1B706 Continued : 

Requires the contractor to 
generate and maintain records 
and data of all tests & insp 
performed. 

2. Requires Equipment Records 
sub items a thru f list 
what these records must con- 
tain. 

Subsystem records are to be 
combined into system records 


13706 All of these records required are re- 
specified on the CIRL/CLRL. The combination 
usually takes the form of a system Log Book. 

|i) with sub-logs for lower order of equipment 
as applicable. Inspection Services, in conjunc-J 
tion with Test Operations, enter that each 
action is performed, inspection personnel 
performing the inspection, witnessing the 
tests performed by test operators, and noting 
that results did/did not meet specifications. 

If the latter, corrective action is initiated. 
QA function is to assure that Logs are properly) 


compiled and maitnained current, and that all 
inspection and Test actions specified have beer) 
carried out. 

Recommend: Re-write to define QA function 


13707 . Contractor Quality 
Assurance Actions. 


specifically., 

1B707 . Stet. All Items Listed. This is a 


Cites QA activity prior to 
inspection and Test operation^ 
Cites QA action during Test 
Operations . 


properly formatted set of statement of 
specific QA responsbility. It is directly 
) applicable to the Quality Assurance organ- 
ization maintained by a contractor, and details 
WHAT is to be done, WHEN , and by WHOM. If NASA 
elects to re-write this section of the HDBK 
recommend this format and approach be used. 


Chapter 8 . Non-Conforming 


Article & Material Control 

Includ 

es: 

IbBooT 

Statement of Policy 

IbBcJI. 

Specifies Documentat- 

1B802 

ion Rq'd & Contents 

Remedial & Preventive 


Action: Cites non 
conformance diagnosis 
methods of remedial 
action implementation 
and notification 

13803. Initial Review Disoos- 


itions . Item la . Covers Rework, 
Item b Covers Scrap. 


Item c . Covers Return to Sup- 
plier, 

Item 2 Covers Review of items 


dispositioned without MR3 
action. 
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Chapter "Is Now" 


Chapter "To 3e" 


1B803 Contn'd. 


Item 3 . Requires Initial 
Dispositions to be documented 

1B80U Material Review Board. 


The entire chapter adequately defines the 
membership, responsibilities, activities, 
and dispositional activities of a material 
review board, instituted for handling non 
conformances . 

As such it is a highly formal activity, and 


Sect. l Defines Membership 
Sect 2. Defines Ra sponsibilit|/pas intended originally to apply to large 


Sect j 3 . Define s Dispositions 


1B803 Written Requests for 


NASA Contracting Officer 
Approval : Coverse conditions 
under which approval must be 
sought for dispositions where 
the contractor recommends 
Repair or USE-AS-IS 

1B80£. Supplier MB 
Permits MRS activity to be 
delegate to suppliers, with 
NASA approval. 


Chapter 9 • Metrology Controls 
Includes: 


13900. General: Contractor 
to establish & maintain 
metrology system to evidence 
quality conformance. 

1B901 Acceptance : Mandates 
conditions under which con- 
tractor is to check stds 
prior to acceptance. 

1B902. Svaluati on : All stds 
(incl auto check out equip) 
to he verified for: 


scale, high financial involvement programs. 

Its chief problem is that it can, and does 
inject delays into program schedules, by the 
mere fact that a board must be convened, each 
time dispositions require handling, as most 
programs cannot afford a direct full-time board| 
Secondly, Contracting Officer approval is re- 
quired under several conditions, and this can 
also inject delays. In practice, contractors 
are required to formulate and maintain a 
Quality Assurance Manual. This document covers) 
the totality of Quality Assurnace Functions 
and responsibilities. Usually, it contains 
MRB and non-conformance handling procedures 
tailored to a FULL Conformance , Limited 
Conformance , and Informal Activity set of 


scales. Based upon prior contractual agree- 
ments, one of these alternatives is selected. 
Years of Program experience has proved the 
effectiveness of the second two levels, which 
are dictated by the complexity and dollar 
value of a program. 

Recommend: Chanter be re-written to allow 


maximum informality, minimum documentation, 
and rapidity of disposition in accordance with 
proved program scaled methods. 
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Chapter "Is Now 11 


1B902 Cont'd ; 

1. Accuracy of measurement 
of desired characteristics 

2. hadware configuration 
compatibility 

3. Correctness of Operating 
Instructions. 

"Records to be kept of eval'ns 

13903 Article or Mat'l 
Measurement Process. 


Mandates random & systematic 
errors in article/mat '1 meas- 
urement process not to exceed 
■40/3 max of article tolerance 
Excp'.ns, to be authorized by 
NASA. 

IB 9 OI 4 Calibration Measurement 
Process: Mandates randon & 


systematic erros not to ex- 
ceed 25$ of tolerance of par- 
ameter measured. NASA appro 
for excp'ns. 

1B905 Calibration Controls : 


Specifies a 7 point control 
process & permits internal or 
external facilities for 
calibration of stds used by 
contractor. Controls are 
Traceability, Handling/ stcrag4 
i/D & Labelling, Calib Inter 
val. Recall System, Calib 
Records and their contents. 

1B906 Environmental Rqmts 

Specifies environmental 
rqmts compatible with accuracy 
of stds. 

1B907. Remedial & Preventive 


Action : non conforming stds 


Chapter "To Be” 


Chapter 9 ' All of the requirements of this 
chapter are valid as such. All aerospace 
contractors and suppliers are familiar with 
these requirements/controls, and these are 
the subject of both Federal Stds, as well as 
DoD tri-service stds. Furthe? each contractor, 
and supplier is obliged to include in his 
Quality Manual, a detailed description of his 
Measurements & Standards facilities, whether 
his own, or those of externally procured origin 
The directive of this chapter apply to the 
Measurements and Standards activity, and check 
to assure that all functions are carried out isj 
customarily perofmea by Inspection Services . 

[The specific role of Quality Assurance is once 
again that of a check and balance to ensure 
proper execution of these requirements takes 
place . 

Recommend : Delete this entire chapter 


Alternatively : Describe this activity once, in 


a standard Quality Program Plan, subject to 
NASA approval. Update description periodically 
as necessary. This will save, as a minimum the 
expense of describing the entire system every 
time a quality program plan is generated and 
submitted in response to every single RFP. 


REPRODUCIBILITY OF ? : 
ORIGINAL PAGE IS P ORf. 


& hdwre measured by them shall 
be subjected to these action 


Chapter 10 : Stamp Controls : 
Mandates a 6 point controls 
system with written proc'drs 
for stamp control. 


Suggest Deletion: In force universally for 1$ 
years, part of every contractor's approved 
Manual of Quality & Inspection Practices. 

For 1B1001 Stamp Restriction: Stst (No MSA) 
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ling, Packaging, Packing & 
Shipping , 

1B1100 . Handling & Storage 

Mandates: Protectiion against 
handling damage, special hand 
ling isntructions as rq’d, 
evidence of periodic proof 
testing of handling equip 'mt . 
2. Suitable storage precaut- 
ions & measures to guard 
against deterioration & dam- 
age, as well as date indicato^ 
for critical life items. 


have specifications regulating contractor 
performance of these tasks, as well as each 
task being specified in either a contract 


1B1101 . Preservation, Mark- 


ing i k Labelling, 

<■ ! Ill I— 


Packaging 


& Packing: 

Item 1 Covers Preservation 
rqmts . 


Item 2 Covers Marking & Labely orme d as specified. 


statement of work SOW, ora Work Breakdown 
structure (VJ3S ) or both. In practice these _ 
tasks are undertaken by specific organizations 
within a contractor’s org structure, such as 
Materials Handling Engineering, Packaging 
Engineering, and Shipping. Frequent lyjthese 
orgs are lowers echelons of a formal engineer- 
ing Support organization. Regardless of the . 
organizational structure, the tasks are require, 
•s)-d contractually to be described and costed 
and contractors have suitable specifications 
and instruction manuals for the regulation of 
this work. Specifically, Inspection services 
checks each step in these tasks/processes, 
against spec/ procedures provided. The role of 
QA is once more a surveillance function, and 
signature to certify that the tasks are perf- 


ing rqmts. 

Item 3 Covers Packaging. 
TtSTU Covers Packing Rqmts. 


w -~'zr . 

Recommend: This paragraph be delated as 


131102 Shipping j 


Item 1 Mandates a 7 point 
a-thru-g shipping control 
set of rqmts. 

Item 2. Do cumentation Package 
Mandates that a contractor 
include a complete doc ' mtn 
package with his shipment . 
Specifies in general terms 
req'd contents of package. 

Chapter 12 . Sampling Plans ; 
Statistical Sampling & Anal- 


superfluous . 


ysis . 

131200 Sampling Plans: 

Covers conditions under which 
MASA will permit contractor 
use of sampling plans. 

1B1201 : Same coverage for use 
of statistical analysis 


Chapter 12. Statistical Quality Control is a 
well established methodology. NASA SOWS rout- 
inely require a contractor to recommend the use 
of sampling plans & statistical techniques^ all 
of which are covered in many DoD, NASA, -!c ^ed 1 
specifications, & in contractors QC Manuals. 
Recommend : Deletion of the chapter as super- 


f luous . In practice few contractor space ^prog- 
rams have large populations of items. Inference. 
is usu a ^l'/’ made by the "Small Samples" technique 
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Chapter "Is Non" 


Chapter "To Be" 


Chapter 13 . 
Control: 


Gov't Property 


Covers : 


1B130Q Contractor's Respons - 
ibility : Defines the responsib 
ility of the contractor to 
account for, and maintain in 
good order, all C-ov't Furnish 
ed Property. Further cites 
[a 5 point procedure & action 
list to be used in effecting 
this task. 

1B1301 . Unsuitable Gov't P rop- 
Mandates that Gov't 


Chanter 13 . All aerospace contractors and 
suppliers are familiar with this mandate. 
Further, the mandate is made an article of 
contractual obligation and compliance is 
to be demonstrated by the contractor's account 
'ing system. 

Recommend : This chapter be deleted from the NHB 


I as superfluous. 


be notified of any property 
that contractor finds to be 
unsuitable for use for any 
reason. No dispostion is to 
be made without Gov't per- 
mission. 


Appdx. A . Quality Progra m 


Document Corss Reference Inde 


References each Quality Doc't 
to Chapter, by para number 
and page number of NHB 5300. 
1|(1B) ko Documents in all, 
by count. No sub-tiers incl- 
uded. 

Appdx. B . Glossary of Terms: 


Defines terminology used in 
'NHB 5300.1i(13) 


Appdx. A . Stet Pro-Tern . 

Should NASA elect to re-write NHB 5300. U( IB) 
which would in all probability eliminate the 
need for some of the quality documents, this 
appendix would require alteration accordingly, 


Appdx 3 : Stet Pro-Tern . 

Should revision of this document result in 
the use of more, less, or different terms 
from those listed and defined, the appdx. 
will require revision accordingly. 
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6. Cost Estimate Associated with Use of NH3 $300»MlB) 

6.1 Basis for estimates : Appendix A of the Quality NHB lists 1*0 documents, 
any or all of which may be mandated as deliverable under the contractual 
requirements of a typical NASA Space Program. The Appendix has been included 
in this E.M. to provide a ready reference, and forms the text of pages 28 & 29. 
Each of the documents listed requires some preparation time, and each document 
furnishes a record, or an outcome of a specific Quality Assurance task under- 
taken as part of the assurance workload. While LMSC has performed relatively 
few contracts for NASA, the NASA documents are also required in either identical 
form, or similar form, by USA? contracts; performance on which forms the bulk 
of LMSC SSD experience in the design and fabrication of high quality, reliable 
space vehicles. The documents listed, together with the tasks to which they 
relate, comprise the quality assurance programs undertaken to date, to a large 
extend. Under certain assumptions and provisos, the documents form the basis 
for these estimates which follow. 

6,2. Estimate Constraints : It cannot be emphasized to strongly that the estimate 
represent the experience of one aerospace contractor, LMSC. The estimates are 

•j f| 

under no circumstances to be considered xnxxccrsag of the magnitude of quality 
expenditures which may be experienced on future vehicle contracts. They are 
useful only in that they afford some measure of what compliance with the require 
ments of NHB 5300. U(1B) have cost NASA historically. 

6.3 Assumptions for Estimates: The following assumptions have been made: 

(a) Estimates are made on the basis of assumed full compliance 
with all of the requirements of NHB 5300. U( 13) 

(b) Assumption is that all of the documents listed in Appdx.A 
have been prepared for a given program (or a similar set 
of documents) 

(c) Many of these documents listed in Appdx. have specific NASA 
formats, and in many cases NASA forms are supplied to a contract- 
or. Assumption is that time to prepare/time to complete a 
document; i.e. fill it out, are interchangeable as a basis for 
estimates . 


I’flMti^'fVirif I'm 
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APPENDIX A: QUALITY PROGRAM DOCUMENT 
CROSS-REFERENCE INDEX 


Page 

Document 

Par. 

2-2 

Training Documents 

1B202-1 

2-2 

Training and Certification Records 

1B202-4 

2-2 

Quality Information 

1B203 

2-3 

Quality Status Report 

1B204 

2-3 

Quality Program Audit Reports 

1 B 20 5 - 3 

2-3 

Quality Program Plan 

1B206 

2-4 

Policies and Procedures 

IB 20 6- 2b 

3-1 

Technical Documents 

1B300 

3-1 

Document Review 

1B300-2 

3-1 

Change Control System Document 

1B302 

4-2 

Identification List 

1B404 

5-1 

Quality Records 

1B501-1 

5-1 

Pre-Award Survey Results 

1B501-2 

5- 1 

Procurement Documents 

, lb502- 1 

5-6 

Receiving Records 

1B506 

5-6 

Post-Award Survey Schedules 

1B508-1 

5-6 

Post-Award Survey Results 

1B508-3 

6-1 

Fabrication Documents 

1B600 

6-2 

Process Control Procedures 

1B603-2 

6-2 

Equipment Certification Records 

' 1B603-3 








. NHB 5 300. It (13) Appendix A.Cont'd 


Page 

Document 

6-2 

Workmanship Standards 

7-1 

Inspection and Test Planning 

7-1 

Test Specifications 

7-1 

Inspection and Test Procedures 

7-2 

End-Item Inspection and Test 
Specifications and Procedures 

7-5 

End-Item Inspection and Test Report 

7-5 

Inspection and Test Records and Data 

7-5 

j ■ . . 

Equipment Records 

8-1 

Nonconformance Documentation 

8-4 

Written Requests for NASA 
Contracting Officer Approval 

9-1 

Procedures for Measurement Processes 

9-1 

Results of Evaluations 

9-2 

Calibration Records 

10-1 

Stamp Control Procedures 

11-1 

Handling Instructions 

11-1 

Storage Procedures 

11-1 

Packaging Procedures and Instructions 

11-2 

Documentation Package 

12-1 

Sampling Plans 

13-1 

Government Property Records 


Par. 

1B604 

1B701 

1B702 

1B703 

1B704 

1B705-7 

1B706-1 

1B706-2 

1B801 

1B805 
1B900 
1B902-3 
1B905-7 
IB 1000 
IB 1 100- 1 
IB 1 100-2 
1B1 101-3 
IB 1 102-2 
IB 1 200 
1B1300-4 
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6.i| Tabulations of Quality Cost Estimates/Ranges : The tabulations which 

follow 1 reflect LMSC SSD & R&D program experience in the domain of quality- 
costs. Table 1. is a compilation of general quality cost statistics, most 
of which have been published in other E.Ms on the subject of Low Cost Managem- 
ement Practices. ; These data illustrate the range of costs to be expected when 
NHB 5300. Ii (IB) or other equivalent document such as MIL-Q-9858A is fully 
implemented during a space vehicle program. 

6.I4.I. Tabulation 1; 


Quality Programs) General Statistics 



General Quality Program Costs/Ranges, (in Percent Total Program Costs) 

H 

For typical space programs in the 50-100 million cost range Quality 
Costs range from 7-115 TPC. 

B 

Quality Costs for typical space programs range from 3h-U55 of Manufact- 
uring costs 

- 

1 

Typical Space Program . Quality Engineering Costs range 23-375 of • 
Quality Program Costs (QPC) 

(Typical Space Program Inspection Services Costs range 63-775 of QPC 
Typical ratio; QE to Inspection 1:3 approx. 

1 

Economies possible: Atmosphere Explorer (RCA/GSFC) Quality Costs: 

3.85 TPC. 

Economies - — ■ 

Attributed to: Integration of Quality/lnspection, Reliability & Config- 
uration Management Effort and Planning. 


6 .I 4 . 2 . Quality Documentation Costs ;Based on the Quality Program documents List 
which forms Appdx. A. of NHB 5300»U(13) LMSC costs to compare/compile quality 
documents have been tabulated. All these are average values and are given in 
manhours as the basic cost compilation/collection unit. Again, these values re- 
present average charges, and times allowed to quality program personnel to 
perform the tasks of preparation/compilation of the documents. 
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Quality Document Title Motion 

brientation 


* QE 

Insp. 

Mh to Prepare /comoile 

Training Documents (prepare) 

X 

X 

Approx UO hrs each 

Training & Certification Records 


X 

Approx 20 hrs per case 

Quality Information Reports (prepare) 

X 


Approx 20 hrs each 

Quality Status Reports (prepare) 

X 


Approx 20 hrs each 

Quality Program Audits (prepare) 

X 

X 

1+0-80 hrs each depending 

Quality Program Plan (prepare) 

X 


on program size. 

1+0-160 hours, depends on 

Quality Policies 1 Procedures. 

X 

X 

program size/complexity 
30-1+0 mh each issue. 

Technical Documents (review only) 

X 


10-20 hrs, depends on 

Document Review (general documents 

X 


size & complexity. 

2-1+ hrs per issue copy. 

such as P.O.s etc) 



Change Control System Documentation 

X 


10 hrs per change approx 

Identification Lists (review only) 

X 


8 hrs per listing. 

Quality Records (compile & issue) 

X 


20-1+0 mh per record 

Pre -award Surveys (reporting only) 

X 


1+0-60 mh per survey 

Procurement documents (complete pkgs) 

X 


8-16 mh per package. 

Receiving Records, (review/prepare) 

Y 

X 

U— 8 hrs pef issue . 

Post Award Survey Schedules, (prepare 

) X 


8-12 hrs each issue. 

Fabrication Documents (review) 

X 


20-1+0 hrs each pkg. 

Process Control Proc'drs( review) 

X 


U-10 hrs each issue. 

Equipment Certification Records (r9v) 

X 


2-k hrs each issue 

Workmanship Stds. (prepare/Compile) 

X 


20-1+0 hrs per Std. 

Inspection & Test Planning (review 

X 

X 

1+0— 80 hrs per plan 

plan inspections) 
Test Specifications (Review &cert) 

X 


10-20 hrs per spec'n 

Inspection & Test Procedures 

X 

X 

10-20 hrs per proc'dr. 

(prepare Insp'n procs Review Test) 
End-Item Test & Insp'n Procedures 



(prepare Insp'n procs Review Test) 

X 

X 

10-20 hrs per proc'dr. 

Inspection & Test Records & Data 
(review & compile/distribute) 

X 


Assumes a data system 
-20-1+0 hrs per issue/tes 

End Item Insp'n & Test Report (Com- 



pile Insp'n, Review Test Info) 

X 


1+0-60 hrs per each. 

Equipment Records (review/update) 

X 


2-1+ hrs per each. 

Non-conformance Documentation (prep) 

X 

X 

Varies widely, depends 

Written Requests for NASA COR apprcvl 

X 


on discrepancy levels. 
2-1+ hrs per copy. 

(QA prepares for program) 



Proc'drs for Measurement Process (rev 

X 


10-20 hrs per review. 

Results of Evaluations (review) 

X 


2-1+ hrs per issue. 

Calibration Records ( review/update ) 

V 

A 

X 

1 hour per equipment. 

Stamp Control (maintain) 

X 

X 

Overhead Charge 

Handling Instructions (review) 

X 


2 hrs per each instr'n 

Storage Proc'drs. (review) 

X 


2 hrs per each proc'dr. 

Pkging Proc'drs & Instr'ns (review) 

X 


2 hrs per each document 

Documentation Pkg (compile) 

X 


8-16 hrs per package 

Gov't Furnished Prooerty (Maintain) 

X 


2 hrc per item if any. 


Tabulation 2. Quality Documentation Costs. 
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'M 6,ht3 : 1 ^ ity Engineering & Insp ection Programs; Major Tasks Costs ; 

" i S^tSkf 2eTd e nt5L^*ThFri- f ’ (lB)^ Sd°? NASA. 

documentation, two or more of the c-ros^ ^ in m ° re thai1 ° ne cate 2 0i y as regards 

task. The assumption is that these ulL j/perSLeT "StadesT ^ 
a typical program, and a Quality Activitv i-ms* v,^ k j indeed they are. on 
costs are acquired per ta ^" 1VXty I/Bo has been prepared against which 


it :; 


Ref 


.1B206- 

1B501. 


— ajor Quality Tasks; Typical Quality W3S (internal) 


• -, + . ■>*. 

1B701 ' 


1B202'*; 


13701*5 

1B506a 

1B600. 


1B705’) 


lB70lpf 

s 

1B801 / 
1B1101 


IB905 .. 

p.taiifcfT * 

. •' : • 

t<*’ ’rut n'. 


@ — -i 



Quality Management 
— 1 Vendor/Supplier Surveys 

'?^n^| CXiUa , lit5r Doc umentation (Preparation & Updates) 
lB2Qb •. Quality Audits 

Quality Planning (Inspec'n & Test, Work-flows & Plans) 
Quality Liaison (Engrg & Engrg Support Services) 
Quality Liaison (Mfrg, & Mfrg Support Services) 

Quality Liaison (Test Operations, & Test Support Svcs) 
Quality. Data Retrieval, Reduction, & Reporting 
Inspection Services Liaison ' ' ' ' 

MRS & Corrective action Follow-up. 

Measurements. & Stds Lab, Liaison & Control 
Quality Training, & Certification 
■Customer (NASA) Liaison 


1B205 

1B206. 

1B300-. 

1B600 

1B703 

1B706 ■ 

1B703 - 

1B801; 

1B905 

1B202 . 

1B805 


Ref 


(average only) 


Inspection Services Management 
Inspectors Training & Certification 
Quality Engineering Liaison 
Receiving Inspection 
In-Process Inspection 
Mnd-Item/Final Inspection 
Test Surveillance 
Non Conformances Reporting 
Pre-Shipment & DD-250 Inspection Activity 
Calibration Inspection (Measurements & Stds 


£QPC 


1.55 
0.55 
U -oo 
0.35 
1.20 
2. h0 
7. id 
2.20 

3.50 

3.20 

1.50 
0.25 
0.70 
i.5o 


/5TPC 


2.5 


$ TPC 


2i25 
2 J50 
3. ; 20 
12.50 
26 . 1*0 
5.15 
12.50 
3.50 

i 1 * 25 

Lab Liaison) 0.50 


6.0 


T abulation 3 . Quality Engrg & Inspection Major Tasks Costs 
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Section 


SYSTEM ENGINEERING AND INTEGRATION COST IMPACT ANALYSIS 
•1 Background 

The study of Systems Engineering and Integration (SE&l) cost impact on un- 
manned spacecraft program costs was undertaken as a part of the Low Cost 
Program Practices Study. The analyses in this area were to include all the 
systems engineering tasks and costs and the engineering integration activities 
encompassing the software of "paperwork" interfacing, hut eliminating the 
physical/actual hardware integration. 

• 2 Objectives and Approach 

The objectives of this, task was to determine the costs of SE&I activity, the 
functions and practices which contribute to this cost, and to identify and 
recommend alternative practices which could reduce the SE&I costs in the 
future. The approach to this analysis is outlined below: 

o List and describe the typical SE&I functions performed. 

✓ 

o Determine from the historical data, supplemented by estimates 

where necessary, the cost vs function for each function. Summarize' 
totals by SE&I function and program. 

o By comparison with each total program cost (TPC) determine the 
percent of TPC for each SE&I function, percent of each program 
and average for all programs . 

o Propose candidate alternate SE&I approaches which will tend to reduce 
or eliminate some of the costs. 
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o Two separate conditions will "be applied: 

That the modified practices were to he implemented on 
current programs (essentially like the historical program 
used as a data base). 

That modified practices were to he implemented on Shuttle-era 
payloads utilizing modularized spacecraft and refurbishable/ 

f 

reusable standard components and/or modules. 

o Estimate the cost savings in terms of average total program 
cost which would result from implementation of the low-cost 
practices . 

. 3 Data 

The SE&I data sources and availability are shown in Fig. 6-1. As. listed, 
only limited data were obtained at the top level with very few breakdowns 
by function and associated costs. In addition to the programs listed in 
Fig. 6-1, top level systems engineering costs were obtained for several 
Mariner Mars programs but with no insight as to functional breakdown. 

The data for the paperwork integration activity are almost non-existent 
and the only integration costs available have to do predominantly with the 
physical hardware integration, which was not to be included in this SE&I 
analysis. The separation of these integration costs is not possible. It 
appears that most of the engineering integration is changed to design 
engineering and is not separable from that cost category either. Therefore 
the integration part of SE&I effort was eliminated from further analysis. 
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Program 

Agency 

Contr. 

; ERTS 

GSFC 

GE 

Nimbus - G 

GSFC 

GE 

OSO-I 

GSFC 

Hughes 

HEAO - A, B, C 

MSFC 

TRW 

Pioneer-Venus 

ARC 

Hughes 

Pioneer 10, II 

ARC 

TRW 

ATS-F 

GSFC 

Fairch. 

AE 

GSFC 

RCA 

MV A/1 -73 

JPL 

Boeing 

SMS 

GSFC 

Philco 

71-2 

AF 

LMSC 


WBS 

Item 

Assigned 



N = None Y = Yes (data available) (Y) - Yes (some data) 

Fig. 6-1 SE&I Data Sources/Availability 
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The systems engineering data were analyzed at the top level only due to lack 
of recorded detail at functional breakdown levels. 

.4 Lack of Commonality 

The term "systems engineering" is a very broad one encompassing many different 
functions and, as practiced by the space industry, these functions and who 
performs them vary from program to program and company to company depending 
on company' s/agency's organizational structure. 

Fig. 6-2 lists six spacecraft programs and the systems engineering functions 
performed on these programs as found in the reference material studied. 

This is not to be interpreted that some of the other functions listed were 
not performed on a given program, only that a reference was not found 

under systems engineering function discussions. The non-referenced 
functions may have been performed by other engineering organizations , by 
program management staff, or not required on a given program to be performed 
by the specific contractor. Frequently, especially if the program is in . 
final design/production phase, some of the systems engineering functions 
have been performed in prior phases or by the customer agency internally. 

As can be seen from Fig. 6-2, there is little consistency among the six 
programs as to what systems engineering functions were performed, except 
that they all did "allocation of performance design requirements" which is 
one of the basic systems engineering functions . All systems engineering 
functions are listed as defined by MEL-STD-499A in Fig. 6-3. Fig. 6-4 lists 
systems engineering functions defined by MIL-STD-881. There is considerable 
overlap in the two definitions, but there are also differences. 
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Function Performed 


Mission Requirements Analysis 
System Functional Analysis 
Allocation Per formance/Design Reqts. 
Preliminary Design & Sy nthesis 
Logistic Engineering 
Life Cycle Cost A nalysis ~ 
Optimizatio n Tradeoffs 
Production Engin eering, Analysis 
System & Item Configuration Specs 


MIL- 

STD 

499A 

Category 


2.1 


Program (Contractor) 



SE e EE 

!> O c 

CD 


Q- > ET 


X X 
X X 

x x" 



Fig. 6-2 System Engineering Functions Performed on Programs (1 of 2) (Preliminary) 








Function Performed 


System Test Planning 


Engr. Change Analysis/Control 


Technical Reviews 


Prepare/Control Engr. Dwgs. 


Conduct or Assist in Test Prog. 


Perform & Review Breadboard 
& Model Testing 


Ground Test & Flight Data Anal. 


Support Launch & Mission Ops. 


Control Budgets & Schedules 


I nterface Comp. (S/C, LV, Exper. ) 


Program (Contractor) 


MIL- 

Std 

499A 

UlI 

O 

< 

LU 

i 

o 

CO 

co 
r — 

> 

to 

sL 

CD ^ 
r* 

t — 

.2 S 

Category 

< 

nz 

O 


to 

a. > 




Fig. 6-2 System Engineering Functions Performed on Programs (2 of 2) (Preliminary) (Cont. ) 




Nimbus-G 






































1. TECHNICAL PROGRAM PLANNING 
& CONTROL 

1.1 CWBS and Spec. Tree 


1.2 Program Risk Analysis 
(Cost, Schedule, Technical) 


1.3 System Test Planning 


1.4 Decision and Control Process 
(Problem and alternative analysis) 

1.5 Technical Performance Measurement 

1.6 Technical Reviews 

(Sys. Reqts, Sys. Des., Prel. Des., 
Crit. Des.) 

1.7 Subcontractor /Vendor Reviews 

1.8 Work Authorization 

1.9 Documentation Control 

(Per MiL-STD-480, Config. Mgt.) 


Titled "Engineering Management" 


SYSTEM ENGINEERING 

2.1 Mission Reqts. Analysis 

2.2 System Functional Analysis (Mission, 
Test, Production, deployment, support) 

2.3 Allocation of Performance/Design Reqts. 
(to hardware, computer programs, pro- 
cedures, facilities, personnel) 

2.4 Prel. Design and Synthesis (Assure com 
pleteness of perform /design allocations) 

2.5 Logistic Engineering (Logistic support, 
maintenance, repair level analysis) 

2.6 Life Cycle Cost Analysis (Iterative; 
economic impact of design alternatives) 

2.7 Optimization Tradeoffs (Risks, tech 
perform., schedule, life -cycle costs) 

2.8 Production Engineering Analysis 
(Materials, Tools, TestEq., Facilities 
Personnel for Manufacturing) 

2.9 System and Item Configuration Specs 
(Prog ram -peculiar items) 


Fig. 6-3 System Engineering Definition — MIL-STD-499* 



System Definitization and Overall System Design 

Design I ntegrity Analysis 

System Optimization 

Cost/Effectiveness Analysis 

Weight and Balance Analysis 

I ntra-and I nter-system Compatibility Analysis 

Program Requirements for: 

o Reliability, Maintainability, Safety, Survivability 
Human Engineering and Manpower Factors Program 
Preparation of Equipment and Component Performance Specs 
Logistics Support Integration 
Design of Test and Demonstration Plans. 

"Work Breakdown Structures for Defense Materiel Items"; invoked by MIL-STD-499A. 


Fig. 6-4 System Engineering Definition - MLL-STD-881* 
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Figures 6-5 and 6-6 show the WBS breakdowns for systems engineering as 
involved on two NASA programs - Atmospheric Explorer (AE) and HEAO. The 
HEAO definition is much mere comprehensive and includes integration activi- 
ties, the AE breakdown is simpler and includes the three basic systems 
engineering functions only. For other programs, WBS breakdowns for systems 
engineering were not available. The main point of these figures is that 
neither SE&I definitions nor WBS breakdowns are specific or consistent. 

These practices create confusion in systems engineering records from program 
to program as to what was done under that lable and where was the coso 
charged . 
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MIL -STD -499 A 
Category 

201 Mission Analyses 

Orbital analysis, orbital parameters, trajectory plots, 2.1 

orbital perturbations, major operational events, etc. 

202 Systems Requirements 

Overall spacecraft system design and engineering requirement. 2.2 
I ncludes spacecraft system definition and assurance of design 2.3 

and functional compatibility between the various subsystems. 

203 I nterface Requirements 

I nsure interface compatibility between the spacecraft and 2.3 

launch vehicle/shroud, the engineering measurements 
GFE, the GFE range and range rate, and the GFE tape 
recorders. 

* Excerpts from WBS 

Fig. 6-5 AE Systems Engineering (WBS 200)* 
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MIL-STD-499A 


SYSTEM ’-ENGINEERING CATEGORIES 

• Mission & Requi rements Analysis 

• Functional Analysis 

• Requirements Allocation 

• Tradeoff Studies 

• Design Optimization Effectiveness^ 
Analyses 

• Synthesis "]■' 

• Technical Interface Compatibility 

• Logistics Support Analysis 

• Producibility Analysis 

• Generation of Specs 

• Other SE Tasks 

• Preparation of Design Ref. Mission 


1.5, 2 


WORK BREAKDOWN STRUCTURE 


B HEAO SE&I 


B1 Sys. Eng. & Sustaining 
B2 Flight Mechanics 


B3 Electrical Des. Inteqr. 


B4 Mech. Des. Integr. 


B5 Thermal Des. Integr. 


* Excerpt from TRW 26000-901-000, Vof. 1, Book 2 
"Systems Engineering Management" dated April I, 1974 

Fig, 6-6 HEAO - A, B, C System Engineering & Integration Definition* 
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•5 Systems Engineering Costs 

The cost data for contractor systems engineering on six spacecraft programs 
is shown on Fig. 6-7* These costs are actuals as changed by individual 
programs to systems engineering as a general function. In terms of percent 
of total program cost, systems engineering ranges from 2.8$ to 11.4$ as shown 
in Fig. 6-7* The average for this six program sample is 6$. Expanding the 
sample to 11 spacecraft programs, with Mariner Mars and other spacecraft 
programs with less firm systems engineering cost data, the average systems 
engineering cost is 5*4$ of total program cost representing little change 
from the previous 6$ average. 

How accurately the 6$ average represents systems engineering cost contribution 
to a spacecraft program is debatable. The reasons are: 

1. Records from only three programs provide insight below the top cost 
level. 

2. To what degree systems engineering functions were performed by the con- 
tractor on these programs as opposed to other program requirements is 

V unknown; and 

3- How much of systems engineering effort, by definition, was in reality 
charged to SE accounts and how much was charged somewhere else is also 
unknown. 

Figure 6-8 shows the breakdown of systems engineering (SE) costs as incurred 
on the Air Force SESP 71-2 program. Total SE percentage was 11.4$ of program 
cost, composed of 6 . 7 $ due to requirements, 4-5$ due to analyses, and 0.2$ 

attributed to meetings and reviews. Figure 6-9 presents the systems 
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Spacecraft 

Program 

Cognizant 

Agency 

S/C Contract 

i 

System 

Engr. 

Cost 

Total 

Program 

Cost 

Avg. 

Year 

Expend. 

S. E. % 
of 

T. P. C. * 

Type** 

Program 

OSO-I 

GSFC 

Hughes 

$2.772M 

$34. 281M 

71/72 

8.1% 

F/0 

ATS-F 

GSFC 

Fairchild 

4. 3 (E) 

75.4(E) 

71/72 

5. 7% 

F/O 

SMS 

GSFC 

Philco 

3. 117 

42.027 

71/72 

7.49% 

New 

AE 

GSFC 

RCA 

0.856 

21.761 

72/73 

3.9% 

F/O 

MVM-73 
(Part #2 
Only) 

JPL 

Boeing 

1.075 

38.597 

71/72 

2.8% 

. ■’ • • ; j 

New: HI 

71-2 

AF 

LMSC 

' 1.378 

12.026 

70/71 

11.4% 

New: HI 


Average 

* Spacecraft Contractor Program 
**HI = High Inheritence 
(E) Estimated 

r 

o 

Fig. 6-7 Costs of System Engineering 
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SE&I Function 

Cost 

% 

TPC. 

MIL-STD- 

499A 

Category* 

• Program Review, Meetings, Customer Interface 

$. 019M 

0.2% 

1,2 

• Requirements 

$. 812M 

6. 7% 


• Contamination 

$. 286M 

2. 4% 

2.3 

• AGE/GHE Requirements 

.036 

0.3% 

2.3 

• Integrated Test Program Requirements 

.209 

1.7% 

2.3 

• Launch Ops. Support/Test Requirements 

.021 

0.2% 

2.3 

• Launch Ops. Support Orbital Ops. Planning 

.148 

1.2% 

2.3 

• Launch Ops. Support Vehicle Sys. Requirements 

.025 

0.2% 

2.3 

• Mission System Analysis 

.034 

0.3% 

2.3 

• Integrated On-Orbit Ops/ Planning 

.054 

0.4% 

2.3 

§ Analyses 

$. 547M 

4.5% 


9 Orbit Thermo Post-Flight Data 

.103 

0.8% 

2.2 

§ Structural 

.130 

1.1% 

2.2 

• Aero-Mech. 

.211 

1.8% 

2.2 

• Mass Properties 

.050 

0.4% 

2.2 

• Launch Entry Thermo 

.010 

0. 1% 

2.2 

• Gen' 1 Space Tech. Coord/ Control 

.044 

0.3% 

2.2 

Total 

! 

$1.378M 

11.4% 



Fig. 6-8 SE Functions Vs. $ On SESP 71-2 
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Cost 

% 

TPC 

MIL-STD-499A 

Category 

• Systems Engineering 

$1. 184M 

3.5% 

2. 

• Systems Analysis 

.788 

2.3% 

2.2 

• Software Plans and Test Support 

.137 

0.4% 

2.3 

• Software Program and Test Support 

.574 

1.7% 

2.3 

• Program Requirements and Support 

.089 

0.2% 

2.3 

. ' i : " i,; ’ ' ’ : i' • .■••’ . ' ' • : 

Total 

! $2. 772M 

8.1% 



Fig. 6-9 SE Functions Vs $ On OSO-I 
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engineering cost breakdown as reported on the OSO-I program. Total percentage 
is 8.1$ of program cost. Systems engineering plus requirements add up to 
3*7$; systems analysis is shown at 2.3$, and software/test support represent 
the remaining 2.1$. 

On Atmospheric Explorer, the breakdown of systems engineering costs was: 
requirements 3*3$, analyses 0.6$, adding to total 3.9$ of total contractor 
program cost. 

Figure 6-10 presents comparable SE breakdowns for the three programs in terns 
of precent of total contractor program cost. Eliminating the stringent 
payload peculiar contamination control requirements from SESP costs makes 
them fairly comparable to OSO-I breakdowns providing an average for the two 
programs • ~ 

o S.E. requirements 4.2$ Total program cost 

o S.E. analyses fr.3$ Total program cost 

Average S.E. total 8.5$ " » » 

This implies that systems engineering effort due to requirements and that due 
to analyses is about equal, at least on those two programs. The comparison 
with Atmospheric Explorer (AE), shows that very little analysis was done by 
systems engineering and the requirements functions were less costly. This 
is most likely due to the fact that AE is a follow-on program with high 
inheritance and little or no new technology. 

The S.E. costs reported for MVM ’73 (Part 2 of Boeing contract) represent 
2.8$ of spacecraft contract cost (Fig. 6-7). This S.E. cost does not include 
other S.E. functions (by IMSC's definition, not by Boeing's) such as: 
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BREAKDOWN 

SESP 71-2 

OSO-I 

SESP & 
OSO-I AVE. 

AE-IC, D, E) 

S. E. Requirements 

4.4ft* 

4. 1%** 

4.2% 

3.3% 

S. E. Analyses 

4.6% 

4. 0%** 

4.3 % 

0.6% 

Total 

9.0% 

8.1% 

8.5% 

3.9% 


*SESP ADJ. 6.8% - 2.4% for extensive payload peculiar contamination control 
requirements. (See Fig. 6-8) 

** OSO-I breakdowns as allocated from Fig. 6-9. 


Fig. 6-10 Systems Engineering Cost Breakdown by Two Primary Functions 
(As % of Tot. Prog. Cost) 
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0 

Contamination Control Requirements 


0 . 1 $ 

0 

Environmental Requirements 


1-3$ 

0 

Structures & Dynamics Analyses 


1.9$ 

0 

Mass Properties Analyses 


0 . 6 $ 

0 

S/C Configuration & Interface Control 





Total 

4. 2$ 


Adding the basic 2.8$ to the k.2 $ would raise the S.E. effort to 7 . 0 $ of 
MVM '73 Bart 2 program cost. 

There may be other S.E. functions included in MVM '73 subsystem or other 
work packages for which costs are not extractable. (The Part 1 of MVM ’73 
contract with Boeing contained some systems engineering effort.) 

As in some other planetary programs, which average only k.4$ of program cost 
for S.E. functions, the NASA agencies may have performed most of the early 
systems engineering functions in-house and there are no data available on 
that effort. Estimating the total MVM '73 SE&I effort results in over 9 $ 
of total program cost including experiments. 

The above discussion is intended to point out that some systems engineering 
and paperwork integration costs are buried on most programs in other func- 
tional cost categories or charged to subsystem engineering and cannot be 
extracted for analysis. 

Since systems engineering is often done by personnel in program office, an 
attempt was made to combine the costs of program management with systems 
engineering and plot against the total program cost. Figure 6-11 shows these 
data. Many programs do not separate configuration management and data costs 
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• Prog ram Management and; j 
data $ separation N. A. for all points)! 

• Systems engr'g and! , 

'program management - and charges/J 
•functions - confused.! 

• Sum, go od fit at j : 

18M/$1M Program cost) 

ISMS Y/S 


SUM 


BEST 
FIT! . 


/y m 

/'/oso-i 


/mmsi ®) 1 


PROG. MGM'T 
) CONFIG. MGM*T 
i& DATA 




\ 1 / Qiy (SYSTEMS |ENGR r G 

\ / 

\ / 

Hmvm . i 

PBOEINGlS.E. $ 


30! 1 40. L_ 150 

PROGRAM COST ($M) 


Fig. 6-11 Systems Engr f g Charges Vs. Program Cost 
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from program management costs, therefore these are shown combined in Fig. 

6.11. Interestingly, the summed curve provides a fairly good fit for the 
six programs (MVM '73 is an estimated point). The systems engineering and 
the program management cost curves flip-flop supporting the contention that 
different programs charge some S.E. to program management or vice versa. 

Even Atmospheric Explorer is not very far out of line (as shown in Fig. 6-10) 
when its S.E. and program management costs are combined. 

On the average, the sum of systems engineering, program management, configu- 
ration management and data represents 18$ of spacecraft program cost. Con- 
figuration management and data costs are fairly consistent at about 2 $ leaving 
1 6 $ for combined S.E. and program management efforts. From the standpoint 
of direct program manhours, 2 out of every 10 manhours (20$) are spent on 
the combined functions of systems engineering, program management, configu- 
ration management and data. 
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6 SE & I PRACTICES 

Historical SE&I practices coaid neither be identified nor quantified daring 
this study. Meaningful identification and quantification was precluded by 
incomplete and inconsistent practices in the areas of Program Definition, 

REP and WBS preparation and record-keeping. 

6.7 CONCLUSIONS 

• It appears as if MSA is paying from 2.8 to 11 Aj> of each programs 
total cost for something called Systems Engineering (or sometimes 
called System Engineering and Integration) but the definition of that 
something is very vague and the associated cost accounting trails 
are most often even more vague. 

• Very few, if any, meaningful PER's can be developed from the MSA 
and Air Force data IMSC has seen to date : 

Ho commonly accepted definition of SE&I function exist within 
MSA: Efforts requested in RFP's vary greatly. 

Charges for various SE&I functions usually are not recorded 
against each function performed due to mixed MSA VffiS and 
contractor organization reporting structure. 

• MSA’s practices relative to structuring complete WBS's (with one 
and only one place to charge for each SE&I task) must be improved 
before accurate practice/ cost evaluations can be made (based upon 
actuals). 
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• The need for various SE&I tasks vary greatly with each type of Program: 

New programs requiring State-of-the-art advancements 
New programs using only proven technologies 
Follow-on programs with High Inheritance (T&H) 

•..Follow-on programs with Low Inheritance (T&H) 

New programs with standardized components 

Follow-on programs incorporating standardized hardware. 

• IMSC may not be able to develop PER's based -exclusively on historical 
data even if complete historical data from 8 to 10 programs is made 
available, because: 

Cost records tend to be misleading and inaccurate 

Cost record to WBS levels 4, 5 and 6 are needed for this analysis 

but are normally reported to NASA at levels 1 and 2. 

• Some PER’s based upon a combination of historical data and engineer- 
ing estimates may be feasible. 

• Checklists for low-cost and high-cost SE&I practices, with support- 
ing explanatory rationale and pertinent cost estimating parameters could 
be developed in lieu of quantified graphic PER's. 

6.8 RECOMMENDATIONS 

* • Discontinue this effort until more enlightening historical data can be 

made available . 

• Develop a NASA-wide definition of SE&I functions to be performed on 
programs which eliminate or clarify overlaps with other functions 
such as subsystem engineering, management, configuration management 
and data management. 
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* * Develop a standardized WBS for charging SE&I charges in a manner 

which assures they are separated from other functions and in 
sufficient detail as to record variations of costs due to variations 
of SE&I practices from program to program. 

• Include results of the above definition and WBS recommendations 
plus specific instructions for their use in future RFP's. 


* This recommendation was made during the mid-term study review and were 
acted upon by NASA. 
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INTRODUCTION 

Lockheed Missiles and Space Company is curi’ently performing the Low-Cost Program 
Practices study for NASA Headquarters under contract number NAS W-2752. The 
principal objective of the study is to identify those NASA space program practices 
that contribute significantly to program costs. As part of this study an analysis of 
the cost impact of Ground Support Equipment (GSE) has been completed, and the results 


of the analysis are presented in this engineering memorandum (EM). The scope of the 
analysis was constrained by the limitations of the relevant data. 



EM No. LCPP-9 
Date: 1 May 1975 


GROUND SUPPORT EQUIPMENT (GSE) 

COST IMPACT ANALYSIS 

v 

9.1 BACKGROUND 

The GSE (Ground Support Equipment) Cost Impact Analysis was oiie of the 
subtasks in the cost impact analysis area of investigations performed as a part of 
the Low-Cost Program Practices study. ■ NASA interest in GSE costs has been of 
long standing, probably dating to the substantial outlays made in the Saturn/Apollo 
days. In this study, the GSE cost impact analysis was limited to unmanned auto- 
mated spacecraft programs and the GSE practices being applied in that type of 
program. 

J.2 OBJECTIVES AND APPROACH TO GSE COST IMPACT ANALYSIS 

The objectives of this subtask were to define the approximate cost contribution 
of GSE to spacecraft programs, and to identify the cost impacting practices which 
lead to higher or lower GSE costs on a given unmanned spacecraft program. 

The GSE considered in this analysis included all program -peculiar GSE used 
at all locations (contractor and'subcontractor plants, NASA and/or contractor field 
sites, and launch base; up to and including launch). Also included was the govern- 
ment furnished equipment (GFE) GSE used on spacecraft programs. Post-launch 
support equipment used at launch site, tracking stations (data links) , and data 
reduction centers were excluded. 

Typical GSE categories pertinent to a space program are: 

o Test and checkout equipment 

o Manufacturing test equipment 
o 'Simulators- ■ 
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o Test support equipment (test chambers, fixtures) 
o Integrated S/C and systems test equipment 

o Transportation and handling equipment 
o Transporters, trailers, dollies 
o Lifting gear, support fixtures, stands 
o Shipping containers 

o Servicing equipment 

o Expendables conditioning/loading/venting equipment 
o Environmental conditioning/protection equipment 

o Launch control/monitoring equipment 

o Consoles (launch processing equipment) 
o Umbilicals/disconnects 


The approach to this task was to collect historical GSE cost and programmatic 
data by researching past spacecraft programs and by sending out a GSE cost impact 
questionnaire directly to NASA centers and the spacecraft industry. An example of 
the questionnaire and a detailed discussion of the responses are included in the 
appendix. The collected data base was used in implementing the following GSE 
analysis tasks: 

Determine type, quantity, and usage (location, transfers, redundancies) 
of GSE on historical and current spacecraft programs. 

Tabulate historical costs of GSE , with breakdown into major categories 

(and for individual equipment items, where possible). 

Analyze the cost impact of specific practices (typically): 

o Use of commercially-available components 
o Specifications imposed (by customer and contractors) 

0 Degree of standardization: 
o Within a single program 
o Among NASA centers 
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o Use of common equipment for launch vehicle and payload 
o Scheduling of utilization for same GSE item (geographical transfer 
vs redundancy) 

(4) Estimate average cost reduction for alternate GSE low cost practices for: 

a. Current unmanned space program hardware 

b. Modularized future spacecraft with refurbishable/reusable equipment 

1.3 TYPES OF AVAILABLE GSE DATA 

The GSE data sources and their availability are indicated in Fig. 1. As 
revealed by this figure, the data base is sparse and incomplete with overall cost 
data available for nine programs, but with very limited cost detail. 

The GSE questionnaire provided additional insight for six of the listed 
programs as to the GSE practices and cost breakdowns, however the questionnaire 
was not intended for detailed GSE description and cost collection. Some of the GSE 
description was obtained from GSE plans, such as on the HEAD program; and for 
other programs listed on Fig. 1 major GSE categories were extracted with their 
costs from studies done by program contractors for NASA. 

Review of these available data pointed out the lack of commonality in GSE 
categories, descriptions, and grouping of costs. Fig. 2 lists the variable termin- 
ology used by programs in defining the GSE categories. 

Also lacking was the description and quantification of existing or GFE GSE to be 
used by a program. The existing GSE could be substantial and is certainly variable 
from program to program. The GFE GSE contribution to a program also varies 
and its magnitude is significant in most cases. With these two major unknown 
variables affecting a given program's GSE, the data obtained were incomplete and 
difficult to analyze due to lack of’ common (normalized) starting point. 
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Contr- 

Agency ' actor 


Detail Equip. 
GSE Q GSE Categ. 

Sent I Answ. List Desc. 


GSFC 

GE 

Y 

N 

N 

N 

GSFC 

GE 

Y 

N 

Y 

N 

GSFC 

HAC 

Y 

Y 

N 

Y 

MSFC 

TRW 

Y 

Y 

Y 

Y 

Ames 

HAC 

Y 

N 

N 

N 

Ames 

TRW 

Y 

Y 

N 

N 

GSFC 

Fairchild N 

N 

(P) 

(P) 

GSFC 

RCA 

Y 

Y 

(P) 

N 

JPL 

Boeing 

Y 

N 

Y 

N 

GSFC 

Philco 

Y 

Y 

Y 

N 

AF 

LMSC 

Y 

N 

Y 

Y 

AF 

LMSC 

Y 

Y 




Genl. 

Desc. 

GSE 

Plan 

O. A. 

N 

N 

N 

N 

N 

N 

* Y 

N 

Y 

Y 

Y 

Y 

Y 

N 

N 

N 

N 

Y 

N 

N 

Y 

Y 

N 

Y 

N 

N 

(P) 

Y 

N 

Y 

Y 

Y 

Y 



Y 


Cost Data 


Fig. i _ GSE Data Sources and Availability 
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_____ Definition 


Test & Support Equipment 


em Ops Support Equipment 


Subsystem Support Equipment 


Assy, Handling, & Shipping Equipment 


Ground Support Equipment 


Handling & Shipping Equipment 


Systems Test AGE - 1 ntegrated Spacecraft 


Launch Complex AGE 


Ground Handling Equipment 


Systems Test AGE - Experiments 


Servicing Equipment 


Checkout Equipment 


Auxiliary GSE 



Fig 2(a) GSE Definitions Used on Programs (1 of 2) 
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Definition 

MVM 73 

Pioneer- 

Venus 

LU 

< 

O 0 
CO 

OSO-I 

<? 

to 

Z3 

-O 

E 

iz: 

HEAO-A, B 

CVI 

1 

Apollo 

on 

CO 

Transportation Support Equipment 










X 

Special Test Equipment 





- 





X 

Mechanical AGE/BTE 




X 







Electrical AGE/BTE 




X 


• 





Systems Test Equipment 





X 






Experiment Simulator 





X 






Integration Test Equipment 





X 

. 





Cleanliness Equipment 



■ 


X 



• 



Mecha n ica 1 Su ppo rt Eq u i p me nt 






X 

X 




Electrical Support Equipment 






X 

X 




Chamber Equipment 

■ 





X 


HI 



Aerospace Ground Equipment 

■ 


X 


• . ■ • -I 



■ 



Experiment Support Equipment 

■ 






X 




Factory Support Equipment 

•• 

• 






X 





Fig 2(b) - 


GSE Definitions Used on Programs (2 of 2) 
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Therefore, most of the limited GSE analysis was done with the data 
extracted from the GSE questionnaire and supplemented by other data on an as- 
available basis. 

9.4 GSE QUESTIONNAIRE 

The GSE questionnaire responses were obtained from one NASA center and 
six spacecraft program contractors representing six different programs (five 
NASA, one AF). 


The highlights of the GSE questionnaire results are discussed below. 

Based on nine spacecraft programs , the contractor-supplied GSE averages 
4.7% of total program cost. The questionnaire results (6 programs) indicated 
the same percentage, although the detailed responses (4 programs) showed 
contractor supplied GSE at 5. 3% of total program cost. 

In terms of major GSE categories, the breakdown shown in Fig. 3 was 
derived. The contractor supplied GSE percentages are based on six spacecraft 
programs and the GFE GSE percentages are from five programs. In terms of 
GSE cost, the test and checkout equipment is the major GSE category, contributing 
80 to almost 100 percent of contractor supplied and GFE GSE costs respectively. 

Utilizing absolute GSE costs for four spacecraft programs (1971/73 $) and 
the above average percentages, the GSE costs by major cost category were 
extrapolated as shown in Fig. 4. This sample indicates that a contractor supplies 
84% of GSE and 16% is GFE on a spacecraft program. This breakdown does not 
consider the existing GSE in contractors' possession (sunk costs), only the new 
GSE charged to the program or new GFE GSE. Also GSE capitalized by the 
contractor is not included. 

The breakdown for new vs modified GSE costs indicates that about 90% of 
GSE cost is spent on new GSE. The usage of surplus GSE from another program is 
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MAJOR GSE CATEGORY 

HANDLING & TRANSPORT EQUIP. 

SERVICING EQUIPMENT 

TEST & CHECKOUT EQUIPMENT 
(INCL. LAUNCH CONTROL EQUIP. ) 


Contractor Supplied GSE 


GFE GSE 


TOTAL 


* TPC - Total Program Cost 


T 

Ave. % 

% of TPC * 

% Range 

12.6 

0.6 

0-5 

7.6 

0.4 

0-10 

79.8 

3.7 

90-100 

100. 0% 

4.7% 



Fig. 3 - Major GSE Category Costs (Percent) 


Contractor GSE 


GFE GSE 


Total GSE Cost 


MAJOR CATEGORY 

% 

Extrap. $ M 

% 

Extrap. $ M 

$ 

Extrap. $M 

HANDLING & TRANSPORT EQ. 


0.33 

: 2 . o 

0.01 

11.0 

0.34 

SERVICING EQ. 

7.6 

0.20 

2.0 

0.01 

6.8 

0.21 

TEST & CHECKOUT EQ. 

79. 8 

2.07 

96. 0 

0.48. 

82.2 

2.55 

TOTAL 

ioo . 0 % 

' $2. 60M 

100. 0% 

. $0. 50M 

100. 0% 

$3. 10M 


PERCENT OF TOTAL 
PROGRAM COST 

SHARE OF TOTAL GSE 


5.3% 


1 . 0 % 


16% 



100% 


* Based on four spacecraft programs. Excludes all contractor and GFE GSE costs which are 
sunk or capitalized by contractor. 


Fig. 4 - Average GSE Costs Extrapolated by Major Category * 
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limited. Most of the contractor supplied GSE on a program (over 80% in terms of 
cost) is used both at the contractor plant and at the launch base. The GFE GSE is 
primarly used at the launch base only (60% in terms of GFE GSE cost). 

The contractor supplied GSE costs, based on responses from five programs, 
are broken down functionally as shown in Fig. 5, 

For additional GSE questionnaire responses and their analysis see the 
Appendix. 

9-5 COMPARATIVE COSTS OF GSE 

The tabulation of contractor supplied GSE costs as charged to the program 
is presented on Fig. 6. Also tabulated for each program is the percentage of 
total program cost represented by this GSE. This percentage ranges from a 
high of 8. 6% to a low of 3. 1%. 

For some programs further GSE cost breakdown was obtained as actuals 
incurred on a program or as estimates. Fig. 7 is the breakdown of OSO-I GSE 
costs as charged to the program , excluding GSE paid for by the contractor and 

GFE GSE. These costs represent one set of GSE. 

The HE AO GSE costs were extrapolated from manhour estimates prepared 
by the contractor and are shown in Fig. S. These costs represent one set of 
GSE as estimated for the HEAO program program. 

Similar detail was not available on other programs, but a GSE cost summary 
by major GSE category was derived for six spacecraft programs as shown in 
comparable 1975 $ in Fig. 9. Then the costs were allocated by reported percentages 
to the functional cost categories as shown in the lower half of Fig. 9. 
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Figure 5 

Contractor GSE Cost Breakdown 
(Percent) 


% Range Ave. % 


Planning and Requirements 

3-5 

4.4 

DesignEngineering and Development 

10-40 

22.8 

Sustaining Engineering 

0-5 

1.4 

Procurement/Purchase/Lease 

10-50 

32.6 

Manufacture of New GSE 

10-30 

16.5 

Modification of Existing GSE 

3-5 

4.3 

Qualification testing 

0-5 • 

1.0 

Acceptance testing 

0-5 

1.4 

Installation and Checkout 

0-5. 

2 .. 2 

Maintenance 

0-10 

3.2 

Documentation 

5-20 

10.2 

Total GSE 


100.0% 


Source: GSE questionnaire, 5 S/C programs. 
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Spacecraft 


Program 


OSO-1 



AE 

MVM-73 


Cognizant 

Agency 


GSFC 

GSFC 

GSFC 

GSFC 

JPL 


S/C 

Contract 


Hughes 

Fairchild 

Philco 

RCA 

Boeing 


GSE 

Cost ' 
M of $ 

$1.05 
10.1 
1. 985 
.729 

2 '. 067 (2 ^ 


Total 
Program 
Cost 
M of $ 


34.281 
110.3 (4) 
44.2 < 5 > 
21.761 
38.597 


Av. 

Year 

Expend. 


71/72 

71/72 

71/72 

72/73 

71/72 


GSE * 3) 
% of 


T. P. C. 


3.1% 

7.7% 

4.5% 

3.3% 

5.4% 


(4) 

(5) 

"( 2 ) 


(Part 2) 

71-2 AF LMSC .921 12.026 70/71 7.6% 



(1) Excluding test equipment for experiments 

(2) Excluding some subsystem support equipment and system test equipment 

(3) Spacecraft contractor program 
(E) Estimated 

(4) Excluding systems test complex 

(5) Excluding tracking station equipment ( w $810K) 


Fig. 6 - Comparative Costs of Contractor Supplied GSE (as charged to the program) 


LCPP-9 















































LCPP-9 


Figure 7 


OSO-I GSE Costs 
,(1 set, actual costs) 


Actual 
Costs ($M) 


Test & c/o equipment : $ .705M 

System Test Equipment $ ■ 439M 

Command Encoder 
PCM Decommutators (2) 

General Test Equipment 

RF Measurement Equipment 
Power Equipment 

Mobile Test Stand (Pwr & Control Servicing) 


Experiment Simulator 

System Test Cleanliness Equipment 

Test Aids & Aux. Test Equipment 

o Servicing Equipment 

(Observatory TV Fixture 
Commercial Solar Simulator) 

o Handling and Transportation Equip , 

(Covers, guards, storage & shipping 
containers) 

o Technical Manuals & Documentation 

Total Contractor GSE * 

(as charged to program) 

Above costs in basic categories: 

o Test & c/o eq. 
o Servicing eq. 
o Handling & transp. eq. 


.211 
.042 
.013 
$. 101M 


$ .196M 


$. 048M 


$1. 050M (3.1% 


$. 735M 
. 105 
. 210 


$1. 050M 

* Excl. GSE paid for by Hughes & GFE Sigma 5 computers. 
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■ if 

I 

■ 

1975 - | 

($M) | 

$ . 875M I 

$ . 545M ~ J 


.262 
. 052 
■ 016 
$. 125M 


$. 244M 


$, 060M 
TPC) $1. 304M 


$. 913M 
.130 
■ 261 

$1. 304M 
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HEAP GSE COSTS (Est. '75 $) 

(1 set, costs based on manhour allocations and total dollar estimates) 


o Integrated C/O Station (ESE) 

Management .175M 
Engineering .075 
System Test - . 100 


$1. 750 M 


350 M 


Power/Ordnance Console 
ACDS Test Console/RCS 

.270 

Control & Monitor Unit 
Telemetry & Command 

. 290 

Console 

. 080 

RF Console 

.240 

ADPE Console 

.200 

Simulators 

. 185 

Aux. & Launch Supt Equip. 

.070 

Spares 

.065 

o Mechanical Support Equip. (MSE) 

$0. 750 M 

10 Fixtures 

$.408 M 

3 Slings/Hoists 

. 054 

4 Sets 

. 094 

2 Dollies 

. 041 

1 Container i 

.055 

MSE Mgm't & Admin. 

. 098 

Total Contractor GSE 
(as charged to program) 

$ 2.50 M (3 

Above costs broken down into basic categories: 

o Test and c/o equipment 

$1,750 M 

o Servicing equipment 

.250 

o Handling & transport 

.500 

Total * 

$2,500 M 


* Additional GSE paid for by contractor. 


Figure 8 
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Category: 


SMS- 
A, B, C 


Test & C/O Equip. 
Servicing Equip. 

Handl. & Transp. Eq. 

Total Contra ct GSE 
% of TPC 
No. of GSE Sets 


$1,750 $.913 

$2,257 

$.680 

$5,077 

. 250 . 130 

- 

. 045 

.169 

.500 .261 

.209 

. 181 

.395 

$2,500 _$1.304 

$2,466 

$0. 906 

$5,641 

3.1 3.1 

4.5 

3.3 

■8.6 


$1. 044 
. 167 
.064 
$1.275 
7.6 
1 


Work Function: 


Planning & Requirements 

$.100 

$. 065 

$.104 

$. 045 

Engineering & Development 

. 875~ 

.588 

.728 

.091 

Proc. /Purch. /Lease 

.975 

.130 

.383 

.453 

Test 

- 

. 065 

.174 

— 

Manufacturing 

1 

. 261 

.624 

.135 

Inst. & C/O 

> .400 

- . 065 

. 070 

— 

Maintenance 

J 

.065 

.035 

. 091 

Documentation 

. 150 

.065 

.348 

. 091 

Total Contr. GSE 

*$2. 500 

$1,304 

$2,466 

$0. 906 


* Excl. all Post- Launch Equipment (i.e. , Tracking Stations) 


$5. 641 


$1,275 


Contractor Supplied GSE Cost Breakdown by Major Equipment Category and Work Function (Est. 1975 $M) 
: ' ■ S Figure 9 
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9.6 GSE PROGRAM PRACTICES 

9 . 6 . 1 Historical and Low Cost Practices 

From the analysis of the available GSE data it appears that total program GSE 
cost consists of five types: 

1) Contractor supplied GSE cost as charged to the program 

2) Contractor supplied GSE capitalized by the contractor and not charged 
to the program (may or may not be sunk cost) 

_ j : • . ' _ . . ' .. . 

3) New or modified GFE GSE supplied at no cost to the program, but at 
cost to the government 

4) Existing GFE GSE already in contractors possession (sunk cost) 

5) Surplus GSE (may or may not be GFE and its replacement value 
represents sunk cost) 

The contractor supplied GSE which is charged to the program has the most 
cost information available as compared to the other four GSE cost types. However 
even these cost data are aggregated by different contractors in different groupings 
(lack of commonality of definition) resulting in a difficult cost data retrieval 
problem. 

The other types of cost are very rarely known or reported and therefore 
cloud the total GSE analysis problem by obscuring a common point of departure 
from which the analysis has to proceed. On a given program it is not known how 
much GSE sunk cost has been applied in addition to the known contractor supplied 
GSE charged to the program. 

Based on very rough order of magnitude (ROM) estimates for six spacecraft 
programs the following highly subjective average percentages for GSE costs by 
type were derived: 
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1) Contractor supplied GSE charged to .program 

2) Contractor supplied GSE not charged to program 

3) GFE GSE new or modified 

4) GFE GSE sunk cost 

5) Surplus GSE at replacement value 

Total ROM GSE Cost 
(excl. post-launch GSE) 


44% 

16% 

9% 

30% 

1 % 

100% 


Therefore it appears that for a given spacecraft program less than half of 
total (as defined above) GSE value is charged to the program. These programs 
were of the early 1970's vintage performed by mature aerospace industry with 
extensive GSE inheritance. 


The above illustrates the basic practice, both historical and low cost, of 
extensive GSE inheritance being applied to current spacecraft programs resulting 
in contractor supplied GSE as charged to the program averaging at 4. 7% of total 
program cost. 

Another historical and low cost practice implemented by half of the programs 
responding to the questionnaire was little use of GSE redundancy. Some redundancy 
of handling equipment, which is relatively low cost GSE, was reported. 


Cycling of GSE from factory to launch base has been practiced historically 
on most spacecraft programs. Also required by most RFPs has been the maximum 
usage of existing GSE. The responding programs indicate that the existing GSE, 
if available, could be used with little or no maintenance. 

Most programs build GSE to project peculiar specifications and some to 
commercial/industrial specifications. These are less severe than government 
GSE specifications, which were applied to less than half of the spacecraft programs 
analyzed. A minimum GSE cost objective has been stated in 2/3 of the program 
RFPs. 
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9 « 6. 2 Historical and High Cost Practices 

An historical and high cost practice reported by most programs, which 
impacts GSE cost, is the change in GSE requirements primarily due to changes 
in the spacecraft, and some due to program plan and experiment changes. 
Apparently most programs do not make provisions in GSE for S/C growth, 
multiple uses, or further uses. 

Another high cost GSE practice applied by some S/C programs is 
formal acceptance testing of GSE. Some programs indicated no GSE testing 
other than inspection, which appears to be a cost effective practice. 

With respect to "existing" GSE (interpreted by responders as other existing 
GSE not in their possession): little use is made of such GSE except in the case of 
predecessor/follow-on space programs. This is due to the fact that little existing 
GSE design or cost information is available. Half of the responders indicated 
that no data or hardware was "available" on "existing" GSE. Lack of common GSE 
terminology /definition contributes to this problem, and one third of the responders 
indicated that they do not keep a consolidated GSE inventory list. Almost all 
responders said that the contractors and the government (in case of GFE items) 
have no cost values recorded for individual GSE items. 

A major hindering practice to multiple program use of "existing" GSE, in 
addition to lack of information discussed above, is the fact that 98% of GSE value 
is retained and stored at the contractor plant at the end of contract - 58% has 
follow-on usage and 40% no further identified use as reported by responders to 
the GSE questionnaire. (Also reported by responders was the fact that 73% of 
contractor supplied GSE cost is for non-deliverable GSE). 

Above points indicate little multi-usage of GSE within an agency and minimum 
interagency sharing, although individual contractors make maximum use of 
existing GSE in their possession. 
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. 7 CONCLUSIONS AND RECOMMENDATIONS 

From the analysis of limited GSE data, primarily results of the GSE survey 
questionnaire consisting of six spacecraft programs, it appears that there is a 
trend towards low cost practices in the GSE area, The maximum use of existing 
GSE in contractors' possession, application of non-government specifications to new 
GSE, reduced or eliminated GSE testing, and non-redundancy of GSE items all 
point in the direction of minimizing GSE costs. 

Potentially significant additional GSE cost reductions could be obtained by 
increased use of industry/agency-wide existing GSE if more information were to be 
disseminated about if - both detailed design and GSE status/cost data. NASA is 
currently in process of computing an inventory of existing GSE called EVS (Equip- 
ment Visibility System) which is to be computerized and possibly available to all 
NASA agencies, as well as contractors in the near future. Future RFPs should 
call out the use of this GSE inventory and require cost-tradeoffs in cases of 
deviations. Another requirement in design of new GSE should be that it be multi- 
use, tending towards standardization, and incorporate allowance for growth of 
spacecraft. 

The additional estimated savings from maximum use of existing GSE under . 
the above multi-use GSE inventory plan can be expected to reduce the GSE costs 
to about one half of the current average costs. This statement is based on 
comparing the contractor supplied Atmospheric Explorer GSE costs (low cost 
program with maximum use of existing GSE and all commercial specifications 
on new GSE) to average GSE cost of the other spacecraft programs. 

The GSE cost comparison shown in Fig. 9. , broken down functionally, 
indicates that the Atmospheric Explorer had lower GSE planning and requirements 
costs, lower documentation costs and no GSE testing costs. Converting these 
savings into percentages and applying them to the average costs incurred on the 
other three spacecraft shown provides the following GSE savings estimates in 
three areas: 
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Planning and Requirements savings 

47% 

Documentation savings 

50% 

Test savings 

100% 


The above three areas combined result in average GSE savings on the three 
spacecraft programs of about 10%, ranging from 5% to 16%, for the contractor 
supplied GSE costs charged to the program. These savings can be implemented 
now before the existing multi-use GSE inventory becomes a reality and provides 
the significant GSE cost savings discussed above. 
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Date: 1 May 1975 


GROUND SUPPORT EQUIPMENT 
COST IMPACT ANALYSIS 

APPENDIX 

GSE QUESTIONNAIRE RESPONSES AND 
SAMPLE QUESTIONNAIRE 

i 

GSE QUESTIONNAIRE RESPONSES 

Seven responses were received to the GSE Cost Impact Questionnaire sent out 
to spacecraft contractors and NASA centers. The following discussion summarizes 
the answers received encompassing six spacecraft-programs of the 1970' s. The 
summary of the responses follows the seque nee and identification numbers of the 
questions in the questionnaire. In some cases there are minor discrepancies in 
the average quantified responses. These are due to change in sample size, since 
not all of the questions were answered by all the responders. 

GSE QUESTIONNAIRE RESPONSE SUMMARY 
1. 0 Program characteristics 

1. 1 Timing of the 6 spacecraft (S/C) programs was: 
o 1970-74 Program starts 
o 16 months average till S/C design freeze 
o 38 months average till first flight 

o 4.3 months average after S/C design freeze, final GSE design release 
o 8 months average for GSE manufacturing and procurement 
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Considerable S/C technology inheritance on all 6 programs was indicated; 
active subsystems were similar to previous programs and three programs 
had similar structure. 

Ground support plans or philosophy on the responding programs were 
essentially the sane as on previous programs for 67% ; 33% had basically 
different plans. 

Number of equivalent spacecraft to be supported by GSE ranged from 1. 5 to 
7.0; average 3. 5. 

Multiple flight programs had the same spacecraft hardware, and the 
experiments were the same for the majority of programs. 

The complexity of S/C and experiments relative to GSE requirements was 
moderate to complex for both spacecraft and experiments. 

Omitted 

Integration and testing of program hardware was performed on: 

a) spacecraft - 100% by S/C contractor 

b) experiment and spacecraft integration - 83% by S/C contractor - 
17% by associate contractor 

c) Integrated S/C with launch vehicle and facilities - 100% by S/C 
contractor with 33% of the programs indicating customer support. 

Program integration was performed at the spacecraft contractors plant, 
with one program indicating experiment/spacecraft integration taking place 
at the launch base. 
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2. 0 GSE plans and requirements 

2. 1 GSE plan was developed by 83% of the contractors as a part of their proposal. 
On one program (17%) the customer prepared the GSE plan prior to RFP 
release. 


On all NASA programs, the GSE plan included both contractor and GFE items. 
GSE functional requirements were prepared as follows: 


Experiment 

Contractor 

S/C Con- 
tractor Only 

Customer 

Only 

S/C Contractor 
& Customer 

For S/C 

- 

67% 

- 

33% 

For Experiments 

17% 

33% 

50% 

- 

For Integrated S/C 

- 

83% 

- 

17% 

Average 

5% 

61% 

17% 

17% 


o S/C contractor prepared most of the GSE functional requirements in all 
areas except experiments. 

2. 4 GFE (government furnished equipment) GSE was identified by: 


Contractor 

o In RFP 

o In contractor proposal 50% 

o In previous contract 

2. 5 GSE for mission equipment was supplied by: 


Customer 

33% programs 

17% 


o Customer only on 33% programs 
o Experiment associate contractor, 33% programs 

o Customer and exp. associate contractor, 17% programs 
o Spacecraft contractor and exp. associate contractor, 17% programs 
After contract go-ahead, cost impacting changes in GSE requirements and 
their degree were: 
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None Moderate High 

o Change in total program plan l 3 1 

o Change in spacecraft 3 2 

o Changes in experiments 3 

o Schedule and funding j 

2. 7 Contract End Item (CEI) Part 1 Design-to specs were required and 
prepared only for a portion of GSE and 

2. 8 CEI Part 2 spec's were not required *on any of the programs responding. 

3. 0 GSE quantity influences : 

3. 1 Redundant quantities of GSE items were required on 50% of the programs. 
3. 2 The 50% of programs requiring redundant GSE sets indicated the following 
average redundant GSE distribution: 


0 

Handling equipment 

2 . 3 sets 

0 

Shipping containers 

1.0 

0 

Servicing equipment 

1.7 . 

0 

Environmental control equip. 

0.7 

0 

Interface simulators 

1.7 

0 

Test equipment 

1.3 


3. 3 In all cases the program schedule allowed moving and cycling of GSE items 
among factory, integration facility, and launch base. 

3. 4 Complete integration of spacecraft and experiments was accomplished at 
the spacecraft contractors' factory on all NASA programs. In case of the 
AF programs, 80% of this integration was accomplished at the launch base 
and only 20 % at the spacecraft contractor's factory. Integration contractors 
and separate government facilities were not used by the responding programs. 

4 - 0 General specification requirements for GSE 

4. 1 The general specifications/requirements applied to GSE design, manufactur- 
ing, and testing were primarily commercial/indus trial and project peculiar 
specs - average 59% of the GSE dollars. Government specs were applied to 
an average of 41% of the GSE dollars. 
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The operating environmental requirements imposed on the GSE were the 
same for the factory and launch base GSE. 66% of the programs indicated 
imposition of project-peculiar specs, with one program (17%) complying 
with only general commercial specs, and one (17%) program complying 
only with government aircraft and ground equipment specs (MIL-E-5400 or 
MIL-E -16400). 

Project peculiar specs had less severe requirements than MIL specs, as 
reported by 80% of the programs using project peculiar specs. One program 
thought the requirements to be the same, and one program indicated that 
depending on the GSE item the severity of the requirements varied. 

The responses to the question: "What type/degree of testing was applied to 
GSE?” By GSE category, were distributed as follows: 



Qual Test 

Formal Mfg. 

Accept Test Insp & Test 

o Ground Handling & Trans p. 
Equip. 

1 

4 2 

o Ground Servicing' Equip . 

1 

3 2 

o Launch Base Equip. 

1 

4 1 

o Factory Support Equip. 

- 

3-4 

o Other - Electrical c/o 
Equip. 

- 

1 

The above indicates predominance 

of formal acceptance testing in all GSE 


categories except for the factory support equipment. 

On GSE supplied as contract GFE , testing was performed by contractor 
only on 66% of the programs; by customer only on 17% of the programs; 
and by both contractor and customer on 17 % of the programs. Of the 
pi'ograms where contractor only did the GFE -GSE testing, all items were 
tested on 75% of the programs, and on 25% of the programs only some items 
were tested. 

Maximum use of existing GSE was specified in all responding program RFP/ 
contract requirements/specs. 
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4.7 


4.8 


All responding programs made a specific review of spacecraft requirements 
and their impact on GSE complexity, compatibility, and cost. 

The response distribution to the question, "Were provisions required in the 
GSE to anticipate the following, "was: 


o Specific application to later spacecraft of the 
same or similar program series 

o Spacecraft growth (size, weight, performance) 

o General application to similar spacecraft in 
other programs 


Yes 

5 


No 

2 


1 (some) 6 
1 (some) 6 


4. 9 


4.10 


The above responses indicate that GSE is tailored for a specific application/ 
program without much consideration for further uses. 

The GSE performance requirements include primarily a moderate contingency 
allowance as indicated by 50% of the programs. 33% of the programs 
indicated a small contingency allowance, and one program said that the 
contingency allowance varied depending on the particular piece of equipment. 
The responses to the question "For what levels of testing was GSE required?" 
were distributed as follows: 


o Component 
o Black Box or Assembly 
o Subsystem 
o Spaceframe 
o Spacecraft 
o Experiments 
o Integrated Spacecraft 


3 

4 
4 
2 
7 
4 
7 


5. 0 
5. 1 


Commonality of GSE 

The investigations of existing GSE designs and/or equipment for possible 
application were made as follows on the 6 programs: 
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By By 

Contractor Customer Both 

o Before RFP release - 11 

o Prior to contractor proposal 6 - - 

o After contract go-ahead 2 - 2 

The existing GSE data sources used and selected items were (in terms of 
number of programs): 




Source 

Used 

Item(s) 

Selected 

0 

MIL-H BK-300, Tech. Info. File 
on GSE 

— 

— 

0 

Predecessor Program GSE Plans/ 
Lists 

6 

4 

o 

Same Agency GSE Inventory 
List 

3 

- 

o 

Other Agencies GSE Inventory 
List 

1 

- 

o 

NASA -wide List of Space Program 
GSE 

1 

- 

0 

AF List of Space Program GSE 

1 


o 

General Services Admin. Surplus 
Lists 

2 

- - 

0 

Defense Industry Production Equip. 
Catalog (DIPEC) 

1 

1 

0 

Commercial/Industrial Applicable 
Equip. 

4 

. 1 

The above answers indicate that most of the responding programs surveyed 
commercial/industrial applicable equipment and all programs reviewed 
the predecessor program GSE. Most of the programs selected GSE items 


from the predecessor program plans/lists. 
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5.3 


5.4 


5. 5 


5.6 


5.7 


Only two programs (33%) Indicated that the existing GSE lists they reviewed 
contained GSE item costs. Others indicated no, or partial, GSE cost 
information. 

All responders indicated that they reviewed the lists of existing GSE 
for space programs only. They did not look for potential common GSE 
equipment from other areas (i.e. , aircraft, missiles, launch vehicles, 
etc. ). ; 


The status of "existing" GSE with potential application to their program was 
described by the four responding programs (in terms of number of programs 
responding) as: 


Available Unavailable 


o Outline Descriptions 3 

o Design/Performance Spec. Sheet 1 

o Detail Design Data 2 

o "Surplus "Equipment . 2 (some) 1 


Three programs (50%) responded that no data/hardware was available on 
"existing" GSE. 

The usability status for "existing" GSE items was: 


o 

Beady for use immediately 

22% 

o 

Required simple maintenance 

44% 

o 

Required -major overhaul 

22% 

o 

Not available because of schedule 
conflict 

12% 


Actual usage Of some GSE from another program was made by 67% of the 
programs (33% had no usage): 


1 program used $20K in terms of replacement value (4 items) 

1 program used $60K in terms of replacement value (4 items) 

1 program used $123K in terms of replacement value (12 items) 
1 program used minor cost items, value and quantity not indicated. 
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To the question "Does your Agency/Company keep an updated inventory 
list for spacecraft program GSE items?" response was as follows: 

o 

Consolidated-total aerospace 
community (NASA/DOD/ 
Industry)List 

17% 

o 

Consolidated-Agency only .list 

17% 

o 

Consolidated - for single program 
series, list 

17% 

o 

No "consolidated" list 

33% 

o 

Don't know 

16% 

After completion of a program the GSE is disposed 
responders): 

of as follows (No. of 


Yes No 

o 

"Advertised" as available 

4 


- within agency 2 

- within DOD 2 

- within NASA 3 

- to GSA 2 

- other (agency 1 

decision) 


0 

Stored under maintenance 

2 2 

o 

Stored without maintenance 

4 

0 

Scrapped (eventually) 

4 - 

o 

Other - disposed of in accordance 
with ASPR (NASA) 24 

1 : 

From the above responses, the majority said that after completion of a pro- 
gram GSE is'hdvertised" while stored without maintenance, and then eventually 


scrapped. 
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5. 10 All programs responded that the GSE design requirement for common 
usage at both factory and launch base was considered and actually 
implemented on most or all items. 83% of the programs traded off that 
requirement in cost-effectiveness terms. 

5. 11 To the question: "How long (average) is GSE held by a program before 
disposal /'most responders had no specific answer. One program felt that 
it was determined by the agency. One program indicated 18 months as 
the average time GSE is held by a program before advertising and 18 
months before scrappage. 

5. 12 86% of the responders to the question: "Is there any policy or procedure 
for interchange of GSE items between NASA centers or NASA and AF", said 
they did not know. One responder (14%) felt there was a GSE interchange 
policy or procedure between NASA centers. 

5. 13 All responders said that lack of common "definitions" for GSE items causes 
a problem in identifying multiple usage or considering commonality. One 
responder felt it was a minor problem. 

5. 14 The responses to the standard format GSE catalog desireability question 
were divided: 

57% said it should be established (possibly only for GSE which is 
available or will be available soon) 

14% said it should not be established (it would not be cost effective) 

29% had no opinion 

6. 0 Definitions and costs of GSE elements 

6.1 The GSE categories used by 5 programs and the percentage of contractor- 
supplied GSE cost attributable to each category are tabulated below: 
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PROGRAMS 

ABC 


D E 


Ground Handling Eq. 
Transport Suppt. Eq. 

Grd. Servicing Eq. 
Mechanical Supt. Eq. 
Electrical Supt Eq. 

Test & Supt Eq. 

System Test Eq. 

Special Test Eq. 

Test Aids & Aux. Test Eq. 
Bench Test Eq. 

S/S Complete Test Eq, 
Factory Test Equip. 
Launch Complex Eq, 

Other Categories (4) 



30% 6% 

70% 65% 

10% 

41% 

1 % 


29% 29% 


6 % 2 % 
4% 

1 % 20 % 
10 % - 


20% 35% 

4% 

10 % - 
10 % 

35% - 

43% 


Total 100% 100% 100% 100% 100% 

6.2 On the average, the contractor supplied GSE cost is attributable primarily 
to test and checkout equipment (including launch control equipment). It 
constitutes 80% of the GSE cost. Handling and transportation accounts for 
13% and servicing equipment for 7% of GSE cost. One responder indicated 
that 5% of the total contractor supplied GSE cost is due to manuals and 
training. 

6.3 Follows 6.4. 

6.4 Based on responses from 5 programs, the GFE GSE cost is almost entirely 
attributable to test and checkout equipment (ave. 96% of GFE GSE) and 

the remaining 4% is equally divided between the handling and transportation 
category and the servicing equipment category. 

6. 3 and 6. 5 

The relative usage of contractor supplied GSE (question 6.3) and GFE GSE 
(question 6. 5) was responded to by 5 and 3 programs, respectively, 
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6 . 6 


The following tabulation presents the summarized 

responses: 


° £ ^ e / usage of contractor GSE (percent of each contractor GSE category 

5 S/C programs) 6 




Both 



Contractor 

Launch 

Plant 



Plant Onlv 

Base onlv 

& L. B. 

Total 

Handling & Transp. 

Eq. 0. 8% 

8.0% 

91.2% 

100 

Servicing Eq. 

7.5% 

16.2% 

76. 3% 

100 

Test & C/O eq. 

12.9% 

12.0% 

76.0% 

100 

Total Usage 

6 . 8% 

12.1% 

81.1% 

100 

o Contractor GSE was moved and 

over 80% of it was used both places. 

o Ave. usage of GFE GSE (percent of each GFE 
programs) 

GSE category) (3 S/C 




Both 



Contractor 

Launch 

Plant 

Total 


Plant only 

Base only & L. B. 

Handling & Transp. 

Eq. o 

0 

0 

0 

Servicing Eq. 

0 

100.0% 

0 

100. 0% 

Test & C/O Eq. | 

46.7% 

20.0% 

33.3% 

100.0% 


23.3% 60.0% 16.7% 


100 . 0 % 


o GFE GSE was primarily used at Launch Base; there was less GFE GSI 
movement. 

The responses by 5 NASA spacecraft programs were averaged to provide 
the following tabulation of average percent cost of new vs. modified GSE: 


Major GSE Categories 

Handling & Transport Eq. 
Servicing Eq. 

Test & C/O Eq. 

Ave. Total 


Contractor GSE 

GFE GSE 


New 

Mod 

New 

Mod 

Total 

87% 

- 

10% 

3% 

100 

100% 

- 

- 

- 

100 

59% 

18% 

15% 

8% 

100 

82% 

6% 

8% 

4% 

100 
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In summary: 

o New GSE represents 90% of GSE cost 

o Mod GSE represents 10% of GSE cost 

o Contractor GSE repressnts 88% of cost 
o GFE GSE represents 12% of cost 

o Contractor GSE is primarily new; servicing equipment is all new. 

6. 7 Based on 6 programs, the contractor supplied deliverable GSE represented 
27% of GSE cost and the non-deliverable contractor supplied GSE accounted 
for 73%, on the average. 

6. 8 Based on answers from 3 NASA programs, the GFE GSE in contractors' 
possession represented 73% of the GSE and additional GFE GSE accounted 
for 25% of total program GFE , on the average. 

6. 9 Minimum GSE cost was identified as an objective in the RFP on 67% of the 
programs, 33% had no such objective stated. 

6. 10 Actual low-cost for contractor-supplied GSE was achieved by 83% of the 
programs and for GFE GSE by 50% of the programs. 

Based on the 6 responding programs, the GSE cost represented on the 
average 4. 7% of total program cost. The range of GSE cost was from 2% 
to 9% of total program cost. 

6. 11 To this question only 4 spacecraft programs provided a fairly complete 
answer. Based on this small sample, the average costs are: 

o Average spacecraft contractor programs cost is $49. 1M 

o Average contractor supplied GSE cost is $ 2. 6M 

o Average government supplied GSE cost is $ 0. 5M 

6. 12 The disposition of GSE from contractors' plants at the end of contracts was 
accompLshed in the following manner (% of GSE value): 
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Contractor 
Supplied GSE 

GFE 

GSE 

Combined 

Ave. 

o Retained at contractors' 
plants for follow-on 
usage 

63% 

50% 

58% 

o Stored at contractors' 
plants (no further usage 
identified) 

33% 

50% 

40% 

o Shipped to government 
facility 

4% 


2% 




Total 

100% 

100% 

100% 


The sum of the first two categories indicates that 98% of the GSE is 
retained and stored at contractors’ plants. 


Only one program out of the 6 provided contractor-supplied GSE to support 
new facilities. The portion of contractor-supplied GSE used for these 
purposes on this particular program was 4%. 

Based on responses from 5 programs, the following tabulation shows the 
contractor GSE cost breakdown: 



% Range 

Ave. % 

Planning and Requirements 

3-5 

4.4 

Design Engineering & Development 

10-40 

22.8 

Sustaining Engineering 

0-5 

1.4 

Procurement/purchase/lease 

10-50 

32.6 

Manufacture of new GSE 

10-30 

16.5 

Modification of existing GSE 

3-5 

4. 3 ' 

Qualification testing 

0-5 

1 . 0 

Acceptance testing 

0-5 

1.4 

Installation and checkout 

0-5 

2.2 

Maintenance 

0-10 

3.2 

Documentation 

5-20 

10.2 

Total contractor GSE 


100. 0% 
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6. 15 The responding 6 programs did not include any costs associated with 
activities after launch, such as GSE sustaining engineering, except that 
two programs included GSE maintenance costs. 

6. 16 The answers to the question: "Is there a separate cost value recorded for 
each item of GSE on the total GSE list for contractor and GFE GSE?" 
were as follows: 

Yes No 

o Contractor- supplied GSE costs 1 6 

o Government-furnished GSE costs 1 5 
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EM No. LCPP-9 
Date: 1 May 1975 
Appendix 


(SAMPLE) 

QUESTIONNAIRE 

GROUND SUPPORT EQUIPMENT (GSE ) 
COST IMPACT 


Question Topics Covered 

1. Program Characteristics 

2. GSE Hardware Plans. Requirements 

3. GSE Quantity Influences 

4. General Specification Requirements for GSE 

5. Commonality of GSE 

6. Definitions and Costs of GSE 


Note: For background, explanation, and instructions for questionnaire 

see Attachment A. See Attachment B for Definitions . 
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PROGRAM CHARACTERISTICS 


What was the relative timing of spacecraft and GSE development 
and production? 

a) Year (CY) hardware contract awarded? | 


b) Months from contract go-ahead to spacecraft design freeze? 1 




c) Months from contract award to first flight? 


d) Months from spacecraft design freeze to final design release 

from GSE for manufacture or procurement? 


e) Months for manufacture or procurement of GSE? 




W hat was degree of spacecraft technology inheritance from previous 
program? 

a) L — I All new 

») Dspacecraft similar; different mission equipment. 

c ) I [ subsystems similar 

. □ Structure □ Elec . Power □ TT&C 

C] Attitude Control/Propuls ion I I 

d) ! I Hone 

e) D (Other) 


Was the overall gr ound support plan or philosophy (manufacturing/ 
integrAtion/test/launch) similar to an earlier program? 


I — I Essentially same 
CH Basically different 


□ 

□ 


Similar 


- 3 - 
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1.4 


What were the quantities of equivalent spacecraft to be supported 
by GSE ? (show quantity in each block) 


f 

a) Flight 



b ) Qualification 

c ) Protoflight 

d) Spare Subsystem Sets 

e) 

f) 


□ 

□ 

□ 


1.5 If program involved multiple flights, was hardware the same ? 


a) 

Spacecraft ; 

i 1 Same 

1 — 1 Similar 

1 1 Different 

b) 

Experiments; 

L_i Same 

1 1 Different 


Relative to GSE requirements. 

what was the degree 

of complexity 

the 

spacecraft and 

the experiments? 


a) 

Spacecraft ; 

CZ] Simple 

1 1 Moderate 

□ Complex 

b) 

Experiments : 

1 1 Simple 

CD Moderate 

1 1 Complex 


_k_ 
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Who performed integration a nd testing of program hardware? 


Separate i 

Customer Spacecraft Integrating 

- Agency Contractor Contractor Cs ) 


a) Spacecraft (Spaceframe 
plus Subsystems') _ 

□ 

□ 

□ 

b) Experiments with 
Spacecraft 

□ 

r 

n 

c) Integrated Spacecraft 

with Launch Vehicle . 

and Facilities 1 

U 1 
□ 1 

i i 

□ 


Where was in tegration of program performed ? 


Separate 

Customer Spacecraft Integrating Launch 
-Agency Contractor Contractor s ) Base 

a) Spacecraft (Space-- 
frame plus 

Subsystems) □ □ □ □ - 


b) 


Experiments with 
Spacecraft 
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REPRODUCIBILITY' OF Tx, : 



1 


ORIGINAL PAGE IS POOR 
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2. 1 GSE HARDWARE PLANS, KEQIS . 


2.1 Was a GSE Flan developed and when? 


a) Before KFP release? 

b) As part of contractor proposal? 

c) After contract go-ahead? 

d) None 

e) 


By 

Contractor 


□ 

□ 

□ 

.□ 


By . 

Customer 


□ 

□ 

□ 

□ 


2.2 Did the GSE Plan include both Contractor-furnished and CEE items ? 

□ Yes □ No 


2 . 3 By -whom were initial GSE functional requirements prepared* 


By 

Contractor 


a) For Spacecraft 

b) For Experiments 

c) Integrated Spacecraft 

*) 


.□ 

□ 


_□ 


By 

Customer 


□ 

□ 

□ 

n 


2.k Was Government-furnished (GFE) GSE identified* 


a ) In KFP 

b) In Contractor Proposal. 

c) After contract go-ahead 

d ) None _ 

e) : 


B y 

Contractor 


By 

Customer 


□ 


□ 


□ 

□ 

□ 


□ 

□ 


-o~. : 
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Who, su pplied GSE for Mission -Equipment ? 

n 


□ 

n 


Customer 

Spacecraft Contractor 
(Other) 


□ Expo 


riment Associate Contractor 


Wgre the cost-signi ficant changes in GSE requirements after 
c ontract go-ahead due to : 




Degree 

of GSE Cost Impact 


Hone 

Moderate High 

a) 

Change in total program plan 

□ 

□ □' 

b) 

Change in snace craft' 

□ 

□ □ 
□ □ 
□ □ 
□ □ 

c) 

Changes in exneriments 

□ 

d) 


□ 

□ 

e) 



Were CEI (Contract End Tt em) Part 1 (Design-to) Snecs prepared 
for GSE? 


a) 

Before contract gn-ahparl ? 

. n 

b) 

During hardware acquisition? 

n 

<0’ 

Required for portion only of GSE _ 

□ 

d) 

Not reauired 

n 

e) 


n 

Were 

CEI Part 2 Specs required? 

t 1 Yes 


□ 


No 


Fo ^approximately what $ of total dollar value of GSE 7 


- 7 - 
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GSE QUANTITY IjfrFEUENCES 

Werp there concurrent activities which required redundant quantities 
Q f _gSE items at fac tory, integration facility, and/or launch base ? 

CH Yes □ No 


C^dltlon stated in 3.1 required how many total ( duplicate) sets 
2 £ J equipment in following categories : (Enter total qty. of sets) 


a) 

Handling Eauinment 

□ 

t>) 

Shipping Containers 

h 

c) 

Servicing Eauinment 

□ 

d) 

Environmental Control Equipment 

□ 

e) 

Interface Simulators 

.□ 

f) 

Test Equipment . 

□ 

g) 


□ 

h) 


n 

Did 

_fche program schedule permit moving or cycling some items of 

GSE 

among factory, integration facilitv. 

or launch base (in lieu 

of providing multiple Quantities ) ? 



fixes 

o 

□ 
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3A 


Was integration of spacecraft a nd experiments (or mission equx g^ 
rnont^ accomplished at: (Enter approx. <jo accomplished at each location) 


a) Spacecraft Contractor Factory, 
h) Integration Contractor 

c) Separate Government Facility . 

d) Launch Base — — 

e) — 


_□ i 

.□ i 

□ t 

□"* 

□ » 

Total 100 io 
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GENERAL SPECIFICATION RETIREMENTS FOR GSE 


What general purpose specifications or requirements were applied 

\ 

to GSE for design, manufacture, and testing ? (indicate by per- 
centage of total dollars spent on Gi^E . ) 

a) □ Government Specs _ \_ □ 

*o □ Commercial/industrial Specs 

c) EH 


d) 


□ 



What operating environmental requirements were imposed on the GSE 


Factory 

GSE 

a) Aircraft and Ground Equip- 
ment Specs (MIL-E-5^-00 or 
MIL-E-16400) □ 


b ) General Commercial Specs . 

c) Project-Peculiar Specs _ 

d) None 


□ 

□ 


e) 


n 


Launch 

Base 

GSE 


□ 

□ 

□ 

□ 

□ 


If project-peculiar specs were applied to GSE, -were they more qr 
less severe requirements than the MIL specs ? 

EH More CH Less EH 


Same 
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Which type/degree of testing was applied to GSE ? 


Qual, 

Tes.t 


Formal 
Accept . 
Test 


Mfg . 

Inspection 


a) Ground Handling and Transport 

□ 

E 

1 

□ 

h) Ground Servicina .. 

E 

1 ] 

□ 

; ' ! 

□ 

c) Launch Base Eauinment ... 

c 


□ 

! i 

□ 

d) Factory Support Eauinment 

c 


E 


□ 

e) (Other) j 

□ 

! 

□ 


□ 

f) (Other) | 

□ 

r 

L 

□ 

l 

□ 


Was testing performed on GSE supplied as contract GTE accomplish' d? 


1 — J By Contractor 


[ — 1 By Customer 

1 1 On some items 

Eh 

n all items 1 — 1 On Wone 


Was maximum use of existing GSE specified in RFP/Contract 
requirements /specs ? 


□ 


Required 


□ 


□ 


Wot Specified 


Suggested 

Was a specific review of spacecraft requirements and design made 
for impact on GSE complexity, compatibility with existing GSE , 
and cost of GSE? 


I I Yes 

□ _ 


□ 


Wo 


Were provisions required in the GSE to anticipat e: 


a) Application to later (modified) spacecraft 
of the same or similar program series? 

h) Growth of spacecraft in terms of size/ 
weight, performance? 

c) General application to similar spacecraft 
in other programs ? 

d) 

-II- 

LOCKHEED MISSILES & SPACE COMPANY, INC. 


Yes 

Wo 

□ 

□ 


□ 

□ 

□ 

□ 

□ 


r.i.r.v 

v ' T;.: 1 ; . _ : 
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4.9 Did performance requirements for the GSE include : 

a) Small contingency allowance? 

1) Moderate contingency allowance? 
c) Excessive contingency allowance? 

a) : 

4.10 For what levels of testing was GSE required ? 

a) □ Component 

b ) I J Black box or Assembly 

o □ Subsystem 

a ) d] Spaceframe 
e) □ Spacecraft 

Experiments 
Integrated Spacecraft 


D □ 

e) n 
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COMMONALITY OF GSE 


Were invest igations made of existing GSE designs and/or equipment 


a) Before RFP release? 

By 

Contractor 

By 

Customer 

□ 

b) Prior to contractor proposal? „ 

n 

□ 

c) After contract go-ahead? 

□ 

a) 

n 

n 

Which inventory lists . descriptions . 

data sources were used in 

determining potential applicability 

of existing GSE 

designs / 

hardware to, the new project? Were items selected from these 

sources for project usage? 




Source 

Item(s ) 


Used 

Selected 

a) MlL-Hdbk-300, Tech. Info. 

- 


File on GSE 

n 

□ 

b) Predecessor Program GSE 



Plans /Lists 

□ 

□ 

c) Same Agency GSE Inventory List _ 

_□ ^ 

□ 

d) Other Agencies GSE Inventory 



List : 

□ 

□ 

e) NASA-wide List of Space 



Program ORTH 

□ 

n 

f) AF list of Space Program GSE 

□ / ■ 

i — i 
□ 

0 >2 continued on next page) 
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(Continued) 


Source 


l’tem(s ) 



Used 

Selected 

g) General Services Admin. 
Surplus Lists ... 

D 

□ 

h) Defense Industry Production 
Equipment- Catalog (DTPRG) . 

□ 

□ 

i) Commercial/lnd us trial 
Aunlicable Eauipment 

□ 

□ 

j) 

□ 

□ 

approximate 

Did the lists of existing GSE vou 

reviewed contain 

cost of each GSE item? 

□ les 

o 

□ 


With regard to Question. 5 .2 . were 

common inventory lists reviewed 


for potential GSE items in areas other than space programs' 


a) Aircraft 

b) Ground/Air Weapons 

c) Missiles 

d) launch Vehicles 

e) 


□ 

□ 

□ 

□ 

□ 
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If -potential equipment usable for GSE were determined as "existing" 
on another program what was status of actual data/hardware avail- 
ab i lity for reuse on the current program? 


a) 

b) 
o.) 
a) 

e) 

f) 

g) 

h) 


Outline Descriptions 

Design/Performance Spec Sheet 

Detail Design Data 

"Surplus" Equipment 

Active Equipment - 
No Scheduled Usage 

Active Equipment - 
Scheduled Usage 


Available Unavailable 


□ 

□ 

-□ 

□ 

_□ 

□ 

.□ 

□ 

-□ 

□ 

.□ 

□ 


□ 


□ 

n 


For "existing" GSE items, (See 5.5), what was the usability status ? 


a) Ready for use immediately. 

b) Required simple maintenance to "use" status, 

c) Required major overhaul. 

d) Not refurbishable . 

e) Not available because of schedule conflict. 


□ 

□ 

□ 

□ 

□ 

□ 


f ) Unknown 

Was any major GSE item(s) from any other program actually used ? 
(Replacement cost; average cost per item) 

( i — i 


Indicate 


□ 

Approx imate 
Quantity 

H 

of Items 
per each 

□ 

Cost Break 

> 


□ 


Yes 


On 

□ 

□ 

□ 


No 


Item Replacement Cost 

$ 5000 

$ 5000 - 10000 
$ 10000 - 20000 
$ 20000 or more 
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REPRODUCIBILITY OP TJL 
ORIGINAL PAGE IS POOP 


5 .8 Does your Agency/ Company keep an updated Inventory list for space- 
craft program GSE items ? 


a) Consolidated - Total aerospace Community. 

( NASA/ D oD/ Ind us t ry ) 

b) Consolidated NASA (or DoD) only, 
e) Consolidated - Agency only. 

d) Consolidated - For single program series. 

e) No "consolidated" list. 

f) 

5 .9 How is GSE disposed of after completion of a program? 


□ 

□ 

□ 

□ 

□ 

□ 


Yes 


No 


a) "Advertised" as available. 

O Within Agency D Within NASA 

□ Within DoD □ To GSA 

b) Stored under maintenance 


□ n 


c) Stored without maintenance 

d ) Scrapped . 

e ) ■ ■ ; - ■ 


□ 

□ 

D 


□ 


□ 

□ 

□ 

□ 


5.10 Was a requirement that GSE be designed for common usage at both 
factory and launch base : 

Ye s ' " 


No 


a) considered? 

b) traded off for cost-effectiveness? 

c) actually implemented (for some items)? 
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□ 

□ 

□ 

□ 

□ 

□ 
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5.11 


5.12 


5.13 


I 5 . 1 ^ 

. ■ ’ • ' '' •’ 


Hov long (average) is GSE held by •program before ; 

a) advertising availability to other programs? 

b) salvaging or scrapping of equipment? 

c) delivery to other user or agency? 

a) 



Months 



Months 



Months 



Months 


Is there any policy or procedure for interchange of GSE items 
between : 


Yes No 


a) 

NASA Centers ? 



□ 

□ 

b) 

NASA and AF? 



□ 

□ 

c) 

Not known. 

- 


□ 

□ 


Does lack of common "definitions" for GSE items cause a problem in 
identifying multiple usage or commonality considerations ? 

□ Yes □ No 


Do you believe that a GSE catalog should be established (catalog is 
a collection of one-sheet specifications with standard format and 
GSE nomenclature) ? • 

-O' Yes - - | Ino 
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DEFINITIONS AND COSTS OF GSE ELEMENTS 


k ; 


6.1 What categories or groupings of GSE or AGE (titles) vers employed 
to describe the elements of GSE hardware in this program (See 
Appendix B for definitions ) , and what was the percentage of 
contractor-supplied GSE total cost for each ? 


a) Ground Handling Equipment 

b) Transportation Support Equipment 


c) Ground Servicing Equipment 


d) Mechanical Support Equipment 
- □ MSE “ □ MAGE 


e) Electrical Support Equipment 
□ ESE □ EAGE 


f) Test and Support Equipment 

g ) Che cko ut E quipmen t 

h) Test Equipment 

i) System Test Equipment 

j) Special Test Equipment 


k) Test Aids and Auxiliary Test 
Equipment 


l) Bench Test Equipment 
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Category 

Used 


% of 
GSE Cost 


□ 

□ 

p 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

: □ 

□ 

: : : □ 

□ 

□ 
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(Continued) 


Category- 

Used 


i of 

GSE Cost 


m) 

Subsystem/ Component Test 
Equipment 

n 

□ 

n) 

Manufacturing Support Equipment 

□ 

□ 

o) 

Factory Test Equipment 

□ 

□ 

P) 

Launch Complex Equipment 

□ 

□ 

q) 


□ 

□ 

r) 


□ 

□ 

s) 


n 

n 

In 

terms of functional application of 

ccntractor-sunplied GSE 

indicate approximate cost (percentage of 100</« cost 'l 

of GSE to 

each of the following categories. 



a) 

Handling and Transport 


□ t 

b) 

Servicing 


□ ■ 

c) 

Test and Checkout 


□ 


(including launch control equipment) 


a) 


□ 


Total 


100 
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What was relative usage of contractor-furnished GSE (as portions 
of percentages used in 6.2) ? 



Contractor 

Factory- 

Only 

Launch 

Base 

Only 

Both 
Factory 
& L.B . 

Handling and Transport 

□ * 

n $ 

D $ 

Servicing 

□ 

□ 

□ 

Test and Checkout 

□ 

□ 

□ 

(including launch control 





e quipment ) 


□ 


□ □ 




h) 

V V 

100 $ 

In terms of functional application of government -furnished GSE, 
indicate approximate cost (percentage of 
each of the following categories . 

a) • Handling and Transport 

h) Servicing 

c) Test and Checkout 

< 3 ) 


cost) of GSE to 

□ * 

□ 

□ 

□ 


100 


REPRODUCIBILITY OP TB ' 
OER5IEAL PAGE IS POOR 
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What was relative usage of government-furnished GSE (as -portions 


of percentages used in 6.4)? 





Contractor 

Launch 

Both 


Factory 

Base 

Factory 


Only 

Only 

& L.B . 

a) Handling and Transport 

EH $ 

□ it 

□ it 

b) Servicing 

□ 

□ 

□ 

c) Test and Checkout 

□ 

□ 

, □ 

a) 

□ 
1 T 

□ 

□, 

■ - 


V 




100$ 



What percentage of total GSE cost was related to work on new 
equipment or modification of existing equipment ? 


Contractor 
Equipment GEE 


a) 

Handling and Transport 

New - 

Ui : 

n io 



Mod - 

□ $ 

□ < 

b) 

Servicing 

New - 

□ 

□ 



Mod - 

□ 

□ 

c) 

Test and Checkout 

New - 

□ 

□ 



Mod - 

□ 

□ 

a) 


New - 

□ 

□ 



Mod - 

\ 

□ 



100 i 
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6.7 

What was the approximate cost breakdown ( rfo) 

of the contractor- 


supplied equipment (including both development and hardware 


a) Deliverable Contractor GSE 

□ * 


b) Won-Deliverable Contractor GSE 



Total Contractor GSE 

100 1° 

6.8 

What was the relative value of additional govemment-fum ished ggp? 


a) Contractor-supplied 

□ 


b) GEE 

Of. 


Total GSE 

100 $ 

6.9 

Was minimum GSE cost identified as a rirndT’am 

objective in the REP? 


dies d No 


6.io 

Was an actual low-cost for GSE (relative to total program cost') 


achieved for the program? 

- 


Yes 

Wo 


a) Contractor-supplied GSE j_] 

□ 


b ) Government-supplied GSE d 

□ 


What is estimated percentage of GSE? 
(relative to total program cost) 

o< 
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6.11 What were the approximate program costs (excluding post-launch 


operations cost/ vs GSE costs)? 


a) $ 

b) $ 

c) $ 

d) $ 


Total Integrated Spacecraft Program 
Total Spacecraft Contractor Program 
Total Contractor-supplied GSE 
Total Government-supplied GSE 


6.12 Of the total value (100$) of GSE at contractor plant after 
contract completion, what $ was ; 

Contractor- 

Supplied 

a) retained at contractor plant for $ 

use on follow-on contract? ! — I 

b) stored at contractor plant; no 

identified further usage? □ 

c) shipped to government facility? □ 


6.13 What percentage of contractor-supplied GSE was provided to support 
new facilities for this program? 


(Provide in terms of portion of 100$ 
giv<=>n in 6.2) 
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6’ ll+ What was approximate allocation of 100$ contractor GSE costs 
to the following ; 

a) Planning and requirements 

b) Design Engineering and Development - 

c) Sustaining Engineering 

d) Procurement/Purchase/Lease 

e) Manufacture of new flat! 

f) Modification of existing GSE 

g) Qualification Testing 

h) Acceptance Testing 

i) Installation & Checkout (of GSE) 

«j) Maintenance of GSE 

k) Documentation (Drawings, Specs, Oper. Instruct., etc.) 

l) 


m) 


6 ’15 Did your answers to previous cost questions i 


Total 


100 i 


include any costs for 


contractor activities after lau nch,? 

a) Sustaining GSE engineering?'. 

b) Maintenance of GSE? 

<0 • 


Yes 

□ 

□ 

□ 


Wo 

□ 

□ 

□ 


6 - 1-6 Is the re a separate cost value recorded for each item of GSE on 
the total GSE list for : 


Yes 


Ho 


a) Contractor-supplied GSE _ 

b) Government -furnished GSE 


n □ 
□ □ 
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EM No. LCPP-9 
Attachment A 


GSE QUESTIONNAIRE 
SPECIAL MOTES AND INSTRUCTIONS 


(1) Purpose .of Questionnaire 

Answers to this questionnaire are being acquired, assembled, and 
analyzed, for the NASA/HQ, Low Cost Systems Office, by Lockheed 
Missiles and Space Company (IMSC) as part of a current study contract 
on Low Cost Program Practices. 

It is hoped that the answers to this GSE questionnaire and analysis 
of the summarized results will provide NASA/HQ with sufficient base 
for altering to lower cost versions some of existing NASA general 
practices affecting procurement of, design performance requirements 
for, and general usage of GSE. 

(2) Completeness /Correctness of Answers 

It is desired that answers be as correct and meaningful as possible 
without going into minute detail. If actual data are not available, 
estimate judgment answers are preferred to "no answer" . 

( 3 ) Request for Supplemental Data 

Supplemental data pertinent to elaboration of some of your answers 
are desirable and can be attached to the completed questionnaire . 

Data particularly helpful would include copies of (a) GSE Plan, 

(b) a listing of GSE used on the project (Identifying existing, 
modified or new designs and equipment), and possibly (c ) some cost 
breakouts of GSE dollars on major GSE items . 

(4) Limitation on Scope 

For the purposes of this questionnaire, only GSE required for use 
through launch operations is to be included. Those equipment such 
as tracking, data acquisition centers, and operation al control 
centers are beyond the present scope of this survey and a nalysis. 

A-l 
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(5) Use of Separate Questionnaire for Each Program 

It is preferred that a separate questionnaire be used for each space 
program. If general answers covering a series of programs is the only 
response that can be provided, please list all programs used as a 
reference for your answers. 

( 6) Handling of Contacts and Questionnaires 

Contact has been made with your Agency or Company separately on another 
of the study tasks relative to a general PER survey (Program Effects 
Relationships) Questionnaire (covered by document IMSC-D387544) in- 
volving personal interviews. It is hoped that your receipt and 
response to this GSE Questionnaire will be possible in time for the 
PER survey staff from LMSC to pick up the filled-in GSE Questionnaire 
and, perhaps, discuss it with your GSE specialist(s) . 

If personal contact is not possible, it is requested that your filled- 
in questionnaire be mailed, to the address shown under "IMSC Contacts". 

Telecon contacts with IMSC personnel listed to c larify or refine 
questi ons or answers are strongly solicited so that minimum turn- 
around time for completed questionnaire return can be obtained. 


IMSC Contacts : 

a ) Please direct all inquiries or special information by telephone 
to LMSC: ' ~ 

Dale Kannaford Orgn. 6l-50 (408)742-8207 

: Ken Urbach : (4o8) 742-5036 

Wayne Miller (4o8) 743-1845 

^ ) Eor i nformation that m u st be mailed , p lease address to: 

Wayne Miller, Advanced Payload Systems 
Lockheed Missiles & Space Company, Inc , 

Space Systems Division 
Orgn. 6l-50, Bldg. 104 
P. 0. Box 504 

Sunnyvale, California 94o88 
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DEFINITION OF TERMS RELATING TO GROUND SUPPORT EQJIPMENT 
GSE TERMINOLOGY 

Many terms are used throughout Government and Industry to describe 
Ground Support Equipment, elements thereof, and closely-related 
equipment. The following definitions are offered to aid in the 
understanding of and in responding to the Questionnaire. 

Ground Support Equipment (GSE) is a generic term for all support 
equipment required to handle, transport, test, checkout, service, 
maintain, or otherwise support a spacecraft. Sometimes the scope 
of GSE is limited to apply only to equipment used at a launch base; 
it is also sometimes extended to cover control centers and tracking/ 
data acquisition equipment for flight operations on some progr ams - 

In the Low-Cost Program Practices Study "GSE" has been used to describe 
equipment used in the factory that supports the same types of functional 
activities that may be required at the launch base or for transport 
to the launch base. Post-launch support equipment is specifically 
excluded from the GSE Analysis study task because of scope limitations . 

Aerospace Ground Equipment (AGE) is a generic term used primarily by 
the Air Force, but also used on some NASA programs as the equivalent 
of GSE. It is normally, but not always, deliverable on an Air Force 
contract, by the DD-250 process, and is required in order to support 
a spacecraft during acceptance testing, transport, and launch base 
activities thru lift-off. 

Support Equipment (SE) is also a generic term that must be defined 
for a specific program. (For example, on the Shuttle program tool-, 
ing was lumped with SE for the DDT&E phase and defined as that equip- 
ment and tooling required to manufacture, refurbish, test, transport, 
handle, assemble, checkout, service and disassemble the solid rocket 
motors and component parts thereof, including Factory Support Equip- 
ment, Transportation Support Equipment, Special Test Equipment, and 
Tooling. Launch Base GSE was specifically excluded. 
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1.J+ Mechanical Support Equipment (MSE) is that support equipment (GSE, 

SE, or AGE) which is primarily meehanical in nature or function. 
Typical examples include Lift Slings, Spreader Bars, Dollies and 
Transporters , Shipping Containers, Air Conditioners, Optical Align- 
ment Equipment, Pneumatic and Hydraulic Servicing Carts, Mass 
Properties Determination Equipment, Solar Panel Deployment Mechanism 
Test Equipment, Spacecraft Access Stands/Ladders and Leak Test 
Equipment. 

1.5 Ground Handling and Transport Equipment (GSHE) is that class of GSE 
or Mechanical Support Equipment used to lift, position, physically 
support, move, protect, store, ship or transport a spacecraft or 
spare major components thereof during final assembly, test, storage, 
shipment, and at the launch base for unloading, inspection, test, and 
mating with the launch vehicle. Typical items of GSHE are Slings, 
Handling Rings, Spreader Bars., Dollies, Transporters, Shipping 
Containers, Impact Recorders and Temperature/Humidity Indicators. 

1.6 Ground Servicing Equipment is that class of GSE or Mechanical Support 
Equipment used to provide hydraulic, pneumatic, propellant and gas 
services and environmental conditioning to a spacecraft during system 
testing or at the launch base. Typical items include "Brako" carts, 
Nitrogen Pressurant Carts, Propellant Transfer Units, Air Conditioning 
Units, and Helium Service Carts. 

1.7 Transportation Support Equipment (TSE) is usually considered as part 
of the Ground Handling Equipment (GSHE) or Ground Handling and 
Transport Equipment sub-catagories of GSE. However, in the shuttle' 
solid rocket motor procurement it was a part of SE, with equivalent 
GSE for launch base functions to be determined. 

1.8 Mechanical AGE (MAGE) is essentially equivalent to Mechanical Support 
Equipment . 
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Electrical Support Equipment (ESE) is that support equipment (GSE, 

SE, or AGE) which is primarily electrical or electronic in nature. 
Typical examples include RF Test Console, Pyrotechnic Test Cable, 
Battery Charger, IR Simulator, Spacecraft Test (breakout or junction) 
Boxes, Blockhouse Console, and Dummy Loads. 

1.10 Electrical. AGE (EAGE) is essentially equivalent to ESE. 

1.11 System Test Equipment (STE) is that test equipment used to perform 
acceptance tests on the total spacecraft system before delivery 
and/or shipment to the launch base. Depending upon where integration 
of the mission equipment/payload with the spacecraft takes place, the 
system test equipment at the factory or assembly plant may or may 
not include elements to test the mission equipment. System Test 
Equipment may be GSE or may be designated as PIE or Special Test 
Equipment, (also abbreviated STE); another common acronym is Vehicle 
Test Equipment (VTE) . 

1.12 Special Test Equipment (STE) is equipment and associated software 
that is of a special nature and required for conducting a specific 
test. The term is so broad that it must be defined for each program. 
It may or may not be equivalent to System Test Equipment, also 
abbreviated STE. 

1.13 Spacecraft Test Equipment is a general term which includes all test 
equipment used to test the Spacecraft, Experiments, Integrated 
Spacecraft or portions thereof. 

1.14 Test and Checkout Equipment is that portion of GSE, both electrical 
and mechanical, which is primarily required to support systems level 
tests at the spacecraft assembly/test factory and to support checkout, 
integration with the launch vehicle, combined launch vehicle checkout 
and countdown at the launch base. It is synonomous with Test and 
Support Equipment. 
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1.15 


1.16 


1.17 


1.18 


1.19 


1.20 


? 


T est and Support Equipment U gse that may be required for 

assembly and test of a spacecraft and/or for servicing and checkout 
at the launch base. 

S ystem Test Complex (STC) is a facility equipped to perform compre- 
hensive system testing at the spacecraft level. It contains 

selected portions of the subsystem GSE or SE which are incorporated 
into the System Test Equipment. 

La unch Complex Equipment (ten) u composed of selected portions or 
duplicates of SIE/STC and additional equipment as may be necessary 
to operate a spacecraft, verify its flight readiness, and condition 
it for launch . It is usually GSE . 

Be^ch Test Equipment (BTF!) is equipment used for acceptance testing 
of government -furnished equipment (GPE) as well as contractor 
procured equipment. It normally applies to the component or black 
box level items of spacecraft and mission equipment. It may also 
be used for "Bench Integration Ttest" when required. 

Te_ st Aids and Auxiliary Test E qu ipment (TA & ATE) are those special 

items and test devices required to check out and troubleshoot a 

spacecraft during final assembly and system test. Typical items 

are breakout boxes, jumpers, cable assembles, dummy squibs (squib 

simulators), multimeters, oscilloscopes, antenna hats, optical 

targets, and dummy loads. Usually these items are not deliverable, 

but there are exceptions, particularly when they are used at the ( 

launch base. There is. some overlap in defintion with ESE for some, 
items . 

C hamber Equipment is a subset of System or Special Test Equipment.' 

It covers items required to support spacecraft tests in environ- 
mental chambers, and is not a term having widespread usage. CS' 
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1.21 Factory Test Equipment (FEE) is usually that equipment needed to 
verify the success of fabrication, assembly, or process at the 
piece part or component level. It is generally common equipment, 
not unique to the needs of a single spacecraft program, and usually 
not. considered as GSE . (See STE) 

1.22 Factory Support Equipment (FSE) is a sub-category under Support 
Equipment, as used on the Shuttle solid rocket program, and is not 
part of GSE. It is primarily electrical, mechanical and other 
support equipment dedicated to the manufacturing process. 

1.23 Manufacturing Support Equipment (MSE) may include some MAGE and EAGE 
to support spacecraft assembly and test functions. Usually does not 
include jigs, fixtures, tooling/machine tools, dies, etc., but may 
include special alignment equipment. Since the abbreviation "MSE" 
is also used for Mechanical Support Equipment care must be exercised 
to relate the usage to that defined for a specific program. 

1.24 Maintenance Ground Equipment (MGE) is a functional subdivision of 
AGE, as used by the USAF, and is that AGE used to support a space- 
craft until launch. It includes the equipment required to handle, 
transport, service, checkout, maintain and safe the spacecraft and' 
is, therefore, equivalent to that portion of GSE that is covered 
within the scope of the Low-Cost Program Practices Study. 

1.25 Operating Ground Equipment (OGE) is the other functional division of 
AGE (See MGE above) and is that AGE which is a functional part of a 
program or system and which operates with the spacecraft as an 
essential operational element. Ground control center equipment and 
tracking stations are OGE. Therefore, OGE is not within the scope 
of the Low-Cost Program Practices Study. 
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1.26 G SE Plan is a document that identifies the support requirements of 
a spacecraft program and the ground support equipment that will be 
utilized to satisfy the functional requirements, GSE lists are a 
basic part of the plan, and identify both design and hardware status 
as well as quantity. A separate list is usually provided for govern- 
ment-furnished equipment (GEE). Common status categories are "existing", 
"modify", and "new". In their simplest form the GSE Plan may consist 
only of GSE lists, while the more elaborate plans will include 
descriptions of each item of GSE, its usage, source, software require- 
ments, interfaces, and also related equipment such as STE, PEE, E3E, 

LCE, STC, et., as well as schedules and procedures for installation 
and checkout of specific items of GSE used with environmental test 
chambers, system test complexes and launch base servicing and control 
facilities. The GSE plan sometimes does not exist as a separate 
document, and on some programs it may be called an "AGE Plan" or 
"Support Plan". 

1,27 — E Cost is the total dollar amount on the acquisition contract for 

GSE. This includes costs for design, modification, manufacture or 
procurement, acceptance test of government-furnished GSE, maintenance, 
calibration, installation, documentation, storage, and disposition . 
at contract completion. Note that the cost or value of government- 
furnished or contractor-owned GSE that may be utilized on the acqui- 
sition contract is not included, and is often not known. 
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'!' ' ' 2 . GENERAL TERMS (NON-GSE) USED IN LCPP STUDY 

Many terms used in the aerospace community do not have consistent 
and commonly accepted definitions. This results in communications 
difficulties and confusion when comparing data from different 
programs, different government agencies, and various aerospace 
companies and divisions of these companies . The following definitions 
are used in the Low-Cost Program Practices Study that LMSC is perform- 
ing for the Low-Cost Systems Office of MSA Headquarters. 

2.1 Agency refers to the actual organization 'having project responsibility, 
management, and procurement authority. Within the NASA it is the 
NASA Centers and the JPLj for Air Force it may be SAMSO, AFCRL, 

ARPA, etc . 

2.2 C ustomer refers to the Agency buying or issuing the contracts for 

a system, and their technical advisors/SETD consultants if applicable. 

2.3 Co ntra ctor refers to the prime contractor for a spacecraft or space- 
frame . 

2A Associate Contractor refers to other procurements on a specific project 
that are sources of GFE to the spacecraft or integrating contractor 
and/or which involve interfaces or interface coordination. For 
example. Experiments are often procured by an Agency from an 
Associate Contractor by separate contract, and are then delivered 

to the Integration Facility as GFE. 

i 

2-5 j ^.t le S rate d , Spacecraft and Costs denotes the combination of a basic 
spacecraft (sometimes referred to by some projects as a "bus”) with 
the experiments or mission equipment. An integrated spacecraft 
j program cost includes all contractor efforts as well as government 

i support costs charged to the program. It excludes post-launch 

' operations for purposes of this Questionnaire . 

1:5 ; .. . - . . ■ 
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2.6 Spacecraft is the basic spaeeframe plus the subsystems required for 
orbital flight and basic support for payloads. Typical subsystems 
provide communications, data storage, telemetry, commands, timing, 
electrical power, thermal control, attitude control and station 
keeping, orbit adjust and sometimes fine pointing. 

2*7 Spaeeframe is the basic structure of a spacecraft, and includes the 
main interconnecting cable harness and the interface with the 
booster. 


2.8 Experiments refer to the scientific instruments (sensors, controllers, 
special on-board data systems, cryostats, etc.) that the spacecraft 
carrier and are the reason or purpose of a particular mission or 
space program. 


2.9 Mission Equipment is used synonymously with "experiments" to denote 
the flight hardware qr "spacecraft payload" for a particular mission. 

2.10 Acquisition Phase or Production Phase refers to the hardware procure- 
ment activity on a space program. Usually the acquisition phase 
contract has been preceded by development activity or definition 
studies that have involved technical and cost trade studies that 
result in the establishment of preliminary designs, program plans 
and schedules, and operational requirements. The amount of design, 
qualification testing, test articles and other development-type 
activities that are included in the acquisition phase contract 
varies widely with the government agencies and the nature and 
maturity of the technology and specific program/project . 


2.11 


Integration Facility is the location at which the physical elements 
are comuined to make up the next higher level system element. Usually, 
in this study, it refers to the location at which the Experiments are 
combined with the Spacecraft to make up the Integrated Spacecraft. 

It may be the spaeeframe or spacecraft contractor factory, a 
separate contractor or government agency facility, or at the launch 
base . 


B-8 


REPRODUOffittlTv 


LOCKHEED MISSILES & SPACE COMPANY. INC. 


LCPP-9 


Integration is the activity of combining basic elements or components 
into a higher level functioning system. Thus, the spacecraft results 
from the integration of a spaceframe with the subsystems; it, in 
turn, is integrated with the experiments or mission equipment to 
provide the complete or integrated spacecraft, which--in turn—is 
integrated with the booster to make up the launch vehicle . 
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INTRODUCTION 


Under contract (NAS W-2752) to NASA Headquarters Low Cost Systems Office LMSC 
is performing the Low-Cost Program Practices study, to identify NASA practices 
that contribute to the high 'cost .of space programs. One task.is the collection and 
analysis of pertinent space program history and cost data for a data bank to support 
the LCPP study and similar studies. This engineering memo presents cost data for 
the Air Force Space Experiments Support Program 71-2, a low-cost program for 
which LMSC was integrating contractor and provided the spacecraft. 
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EM No. LCPP-10 
Date: 15 May 1975 


SPACE EXPERIMENTS SUPPORT PROGRAM 71-2 
(SESP 71-2) COST ANALYSIS 


• 1 OBJECTIVE 

In order to provide NASA LCSO with historical cost data and insight into low 
cost practices the Space Experiment Support Program 71-2 (SESP 71-2), which was 
reported as a low-cost project by Astronautics and Aeronautics in its June 1974 
issue, was cost analyzed. The resulting costs were categorized by subsystem and 
major system level support categories, and a split was made separating the one 
time non-recurring development costs from the recurring unit and operating costs. 

Within the subsystem costs, the costs were further segregated down to the 
black box or major assembly level to provide as much functional detail as possible 
within each black box. 

In all cost allocations' the basic accounting categories of direct labor, over- 
head, material, and other costs were kept intact and direct labor manhours provided. 
These data represent actuals in real year dollars as incurred by the SESP 71-2 
program. 

• 2 SESP 71-2 PROGRAM OVERVIEW 
. 2.1 SESP 71-2 Mission 

The Lockheed Missiles and Space Company (LMSC) was awarded the SESP 
7>2 contract with the go-ahead date of April 1 , 1970. The contract called for LMSC 
to conduct a combined contract definition, acquisition and operational support phase 
for SESP Flight 71-2. The mission of SESP Flight 71-2 was to place an approved 
payload into earth orbit in a manner and configuration necessary to the attainment 
of total payload objectives, and provide operational support for the payload for the 
mission lifetime of 6 months. The payload consisted of the following four GFE 
experiments: 
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(1) Flexible Solar Array (RTD-806) : The RTD-806 solar array 
experiment was a 6 month minimum flight test to demonstrate 
the solar array's dynamic and mechanical behavior and power 
producing capability, and to perform specific reference 
measurements. (Hughes Aircraft Co. ). 

(2) Celestial Infrared Measurement jSAMSO-002) : The purpose 
of the SAMSO-002 experiment was to map a portion of the 
celestial sphere for 3 revs and to demonstrate the life 
capabilities of the sensor cooler system for 5 months 
minimum. (Hughes Aircraft Co. ). 

(3) DASA/ONR Input/Output Experiment (ONR-OOl) : The ONR-OOl 
was an experiment for accumulating data on the D, E, and F 
ionosphere regions, performing radar and VLF propagation 
studies, and studying the behavior of eneigetic electrons iof 

5 months. (LMSC). 

(4) Experiment (NSA-101) : This experiment was a flight test to 
demonstrate and observe the capabilities of the NSA-101 
Command and Interface System. (TRW Systems, Inc.). 

Fig. 2-1 shows an overall view of the SESP 71-2 spacecraft and identifies 
the major system requirements. 

* • j * ' 

The Space Experiments Support Program (SESP) Office, SMTE, Head- 
quarters Space and Missile Systems Organization (SAMSO) , was the overall Air 
Force Management Agency for this contract. The contract provided for General 
Systems Engineering/Technical Direction (GSE/TD) to be performed by the Aero- 
space Corporation under the cognizance of the SESP Office. 

.2.2 LMSC Contract Scope 

The LMSC contract scope included the following contractor effort: 

o Define a design baseline for government approval for the Spacecraft 
System, AGE/GHE, interstage adapter, payload fairing, and soft- 
ware. The launch vehicle system/ascent stage/spacecraft concept 
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• LIFE: 6 MONTHS MINIMUM; HIGHLY RELIABLE 

OPERATION 


, 





Fig. 2-1 - SESP 71-2 System Requirements 
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to be utilized was the Thorad (SLV-2H)/Agena. 
o Plan the detail design, production, test, and support of the 
Spacecraft System, Spacecraft System AGE /G HE, interstage 
adapter, payload fairing, and software, 
o Accomplish the detail design, production, experiment 
integration, test, and support of the Spacecraft System, 

Spacecraft System AGE/GHE, interstage adapter, payload 
fairing, and software. 

o • Accomplish SESP Plight 71-2 Total Mission System Integration. 

The deliverable items were: 

o Spacecraft system 

o Spacecraft system AGE/GHE 

o Interstage adapter 

o Payload fairing 
o Data and reports 

• 2 - 3 Program Time Span and Schedule 

The initial contract consisted of three phases spanning a total of 24 months 
with the launch initially scheduled for end of September 1971: 

o Definition Phase - April 1, 1970 through June 1970 (3 months) 
o Acquisition Phase - July 1970 through September 1971 (15 months) 

. o Operations Phase - October 1971 through March 1972 (6 months) 

The actual launch took place on October 17, 1971, 18. 5 months after go- 
ahead, and the 6 month operations phase was extended twice (6 months each) due 
to the longevity of the spacecraft. The operational support was terminated in 
April 1973 resulting in a 36. 5 month program span. 

Fig. 2-2 shows the major program milestones and activities during the 
SESP 71-2 definition and acquisition phases. Fig. 2-3 presents the SESP 71-2 
integrated mission schedule and Fig. 2-4 details the orbit operations schedule. 
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. 2. 4 General Technical and Other Cost Impacting Data 
. 2.4. 1 Technical 

In general SESP 71-2 was a modified Agena upper stage and integrated 
spacecraft with a dry weight on orbit of 3443 lbs. including 492 lbs. of experiments. 

It consisted of the following subsystems: 

including payload module, forward and aft 
racks of the Agena, payload fairing, and 
booster adapter 

active consisting of the heat exchanger 
coolant system and passive (blanket and paint) 
including Agena Bell engine, propellant tank, 
support structure,' plumbing, and pressurization 
system 

consisting of. solar array, batteries and distribu- 
tion system 

including inertial reference package (IRP) , 
horizon sensors, CMG’s, and Ng attitude 
control system 

consisting of PCM telemetry system, two 
S-band data links, UHF command system, and 
including two tape recorders and 12 antennas 
including experiment supporting and connecting 
hardware for the GEE experiments 

Detailed description of the technical requirements, performance parameters 
and subsystem hardware description is contained in LMSC EM LCPP-7, SESP 71-2 
Program History and Spacecraft System Descriptions. 
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o Structures 

o Thermal control 
o Propulsion 

o Electrical 

o Stabilization and 
controls 

o TT&C 

o Experiment 
Support 
Equipment 






• 2.4.2 Inheritance 

The SESP 71-2 program had a high degree of inheritance in terms of 
existing design, off-the-shelf hardware, and support equipment. Approximately 63% 
of the spacecraft hardware was of existing design, 24% consisted of modified equip- 
ment, and 13% represented new equipment. The ground support equipment was about 
45% existing, 3% GEE, 29% modified, and 23% new (the latter including a high 
proportion of ground handling equipment items). 

As a result of the high degree of inheritance and confidence placed in 
mostly flight qualified hardware, the SESP 71-2 program had minimal test hardware 
(for the new components only) and no complete test vehicles. Fig. 2-5 identifies 
the components which underwent qualification testing and shows the schedule for such. 


For further discussion of inheritance see EM LCPP-11. 


• 2.4.3 Reliability 

The spacecraft reliability requirement was 0. 9 for 6 months. In flight 
the spacecraft far exceeded that, since it operated for more than 18 months. 

■ 2 . 4.4 Government Furnished Equipment (GFE1 

The following major items were furnished as GFE: 

o Final Guidance Constants 

o Test Complex Items at the Contractor's Facility as Defined in 
Systems Test Plan 
o BTL Guideset Series 600 
o BTL Waveguide Converter Kit 

o Experiment Test Articles, Simulators and Handling Equipment 
RTD-806 

SAMSO-002 (Flight Article with Sensor Simulator) 

ONR-OOl (Flight Article without Sensors) 

NSA-101 
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o Flight Experiments 

RTD-806 

SAMSO-002 

ONR-OOX 

NSA-101 

Also furnished as GFE, or at refnrbishment cost to the program, were 
different black boxes and existing test hardware from other programs. This resulted 
in $371 K cost savings to the program. 

.3 HISTORICAL FINANCIAL DATA 

.3.1 Contract Data 

The SESP 71-2 contract was negotiated at a target cost of $8. 525M. It was a 
CPIF contract with fee range from 0 to 14. 5% max. The subcontract portion was 
estimated at 17. 5% and the inplant effort constituted 82. 5% of the work. 

At the closure of the contract the actual cost was $12. 026M with $334K fee 
(2.8%)^ representing a price of $12. 360M. The actual subcontract portion was 14. 5% 
and 85. 5% work was done inplant. - 

.3.2 Cost Data Records 

The financial records used as the source for this cost analysis consisted of 
labor and material cost accounting tapes, subcontract data, and top level WBS 
financial reports. 

The labor and material costs were charged to over 1500 account numbers, 
which represented various levels of detail as far as the black box/ major assembly 
level of breakdown and identification Was concerned. Therefore not all black box/ 
major assembly costs were retrievable from the historical data. Fig. 3-1 tabulation 
shows the number of black boxes and major subcontracts identified in SESP cost 
records. 
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Subsystem. 


Structures 
Thermal Control 
Propulsion 
Electrical Power 
Stabilization & Control 
TT&C 

Experiment Support 


No. of Isolated 
Major Assys. /Black Boxes 


4 

1 

3 

13 

8 

14 

4 


Total 


47 


*Major subcontract - over $100K 


Fig. 3-1 - SESP 71-2 NR&R Cost Allocation to Black Box Level 
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The major problem with accounting records of this type is that, with the 
exception of a few accounts, ho provision is made for the non-recurring (NR)/ 
recurring (R) cost split. In categorizing costs as NR and R requires allocation 
based on some other guidelines. In this allocation, the SESP schedules such as 
shown in Fig. 3-2 in conjunction with the opening and closing dates for each account 
were used as activity time-span guides. Also the nature of the account was 
categorized as containing NR, R, or both types of cost subject to allocation. 


Another problem, apparently inherent in accounting records, has to do with 
the common charges. These are general accounts for common material, common 
manufacturing, common quality assurance, etc. , and accrue costs in general accounts 
which have to be then prorated back to the individual direct accounts. Other common 
accounts were burdens such as material burden, travel, reproduction, and computer 
charges. All of these required proration back to direct accounts. 

• 4 SPECIFIC COST ALLOCATIONS AND CHARGFS 


costs: 


The following basic cost categories were established for SESP 71-2 actual 


o Labor 


o Overhead (O/H) 


o Material & Parts 


o General & Adminis- 
trative (G&A) and 
Other Costs 


actual costs and hours as accumulated 
including overtime (O/T) 
plus allocated common charges 
actual costs as accumulated 
plus allocated common charges 
actual costs as accumulated 
plus subcontract dollars (incl. subcontract fee> 
plus outside production dollars 
plus allocated minor common material 
plus allocated procurement burden 
Actual costs as accumulated 
plus purchased services 
plus travel 
plus reproduction 
plus computer 
16 
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The allocations were made based on these groundrules; 
o Common Changes 

Common manufacturing prorated to manufacturing 
Common quality assurance prorated to Q, A. 

Common technical services prorated to engineering 
Common repair and maintenance prorated to tooling and test equipment 
All common prorations based on direct hours 
o Material Burden and Common Minor Material 

Prorated to material based on direct material dollars 
o Travel 

50% allocated to program management 
50% allocated to engineering 
o Reproduction 

25% allocated to program management 
50% allocated to engineering 
25% allocated to data 
o Computer 

25% allocated to program management 
50%. allocated to engineering 
25% allocated to flight operations 

* Other functional assumptions: 

o Recurring acceptance test - charged to test 

• o In-process inspection - charged to Q. A. 

o In-process factory test - charged to test * 

o Manufacturing test equipment - charged to spacecraft test equipment (STE) 
o Final assembly - charged to system integration 


/ 

} 

< 

) 

it*. ^ 
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SESP 71-2 COSTS 


.5.1 Program Cost Summary, 

The SESP 71-2 non-recurring and recurring costs are summarized in Fig. 5-1. 
Subsystem costs and hours are followed by the system (program) level functional 
breakdown and the fee is indicated. All subsystem and system detailed tables show 
costs only excluding fee. All costs are in real year dollars as incurred by the SESP 
71-2 program during 1970/71 time span with some support and operations costs 

extending into 1972/73. 

Fig. 5-2 shows the actual total annual cost spreads as allocated by subsystem 
and system level cost categories. 

The following subsections present the subsystem costs with isolated black box/ 

' major assembly costs identified. The system (program) level cost tables are shown 

subsequently. 

.5.2 Subsystem Costs 

The subsystem total costs are broken down by functions at the subsystem 
level and by functional costs at the black box/major assembly level. For example, 
the "direction and integration" cost at the black box level contains technical manage- 
ment and integration costs for that box. The given subsystem management and 
integration is shown separately at the subsystem level. The overall system integration 
and program management is shown at the system level in a separate tabulation of 
system level costs. 

.5.2.1 Structures 

The major structural elements were the spacecraft module, the payload 
module, the payload fairing, and the booster adapter. The payload module required 
new design, subsequent redesign due to GFE experiment specification changes, a 
test article, and static and dynamic testing. The other structural elements were of 
essentially existing or slightly modified design. 
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BREAKDOWN 

YEARS 

TOTAL 

1970 

1971 

1972 

1973 

Structures 
Thermal Control 
Propulsion 
Electrical Power 
Stabilization and Control 
TT&C 

Experiment Supt. Equip. 

Subsystems 

| 1026 
136 
30 

869 

323 

1235 

195 

$ 552. 
31 
443 
560 
337 
1210 
78 

$ 11 

7 

11 

8 


$ 1578 
167 
473 
1440 
667 
2456 
281 

38l4 

03211 . 

0 37 


0 7062 

Prog. Mgm't and Data 
System Engineering 
System Test 

Sys. Integration & Final Assy. 

Reliability 

Quality Assurance 

STE 

GSE 

Spares & Logistics 
Launch Ops. & Supt. 

System 

$ 226 
699 
157 
135 
i4o 
125 

315 

97 

.? 309 
618 
, 677 

289 
. 156 
180 
280 
224 
28 
58 

$ 47 
26 

4 
2 

5 

4 

2 

3.27 

$ 2 
32 

1 $ 584 

1343 
834 
428 
298 
3io 
595 
325 
30 
217 

$ 1894 

$2819 

$217 

¥~W~ 

$ 4964 

Total Cost 
Fee , 

Total Program 

$ 5708 
159 

$6030 

167 

$254 

7 

$ 34 
1 

$12026 

334 

$ 5867 

$61.97 

$261 

$ 35 

$12360 
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Fig. 5-3 shows the structures subsystem costs and direct labor hours 
as allocated by NR and R breakdown. The subsystem launch operations support 
identified by subsystem is also shown at the bottom of Fig. 5-3 and is included in 
the non-recurring and recurring unit costs. 

.5.2.2 Thermal Control 

The only costs identified with the SESP 71-2 thermal control subsystem 
were related to the heat exchanger coolant loop system which was a new development 
item. Its non-recurring cost included building and testing of a qualification unit 
which was subsequently refurbished with refurbishment costs charged to the recurring 
unit costs. 


Fig. 5-4 shows the heat exchanger costs. It is suspected that the passive 
thermal control blankets and paint were charged to structures or final assembly. These 
costs were minor and could not be identified from the SESP_records. 

.5.2.3 Propulsion 

The propulsion subsystems costs are presented in Fig. 5-5. The propulsion 
subsystem was an off-the-shelf design utilizing the standard Agena engine and 
propellant tank. The engine was purchased from Bell under a subcontract as shown 
at the bottom of Fig. 5-5. The engine subcontract breakdown shows Bell costs and 

-t 

manhours with the total price charged to SESP entered under "engine - parts and 
material" column. 

The minor propulsion subsystem non-recurring costs represent 
essentially SESP start-up costs. 

.5.2.4 Electrical Power 

The electrical power subsystem and black box costs are tabulated in 
Fig. 5-6. This subsystem consists primarily of an existing design with some major 
modifications of change controller, shunt regulator, and power logic assembly boxes, 
as well as a few minor new components. The solar array was an existing design, but 
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wire harnesses and total system design integration involved considerable design 
effort. The reuse of existing hardware as GFE or at cost of refurbishment was 
estimated to contribute about $55K in savings to the SESP electrical power subsystem 
unit cost. 

The electrical engineering launch operations support for this subsystem 
is included in Fig. 5-6 costs. 

.5.2.5 Stabilization and Control (S&C ) 

This was an off-the-shelf design subsystem With minor modifications of 
flight controls and timer black boxes. This subsystem, more than any other, ; 
benefitted from the reuse of existing hardware. It is estimated that savings of about 

$286K were realized in the S&C subsystem unit cost due to black box refurbishment 



and GFE componentry including most of the attitude control items. 

| The major S&C subcontract for procurement of four control moment 

gyros (CMG's) is shown at the. bottom of Fig. 5-7; identifying subcontractor manhours 
and costs. All the S&C subsystem costs incurred on the SESP 71-2 are also 
presented in Fig. 5-7. 

• 5.2.6 Telemetry, Tracking and Command (TT&C) 

The TT&C subsystem involved several new mission critical blackboxes. 
The Pulse Code Modulation Unit type 5 (PCM-5) and Experiment Interface Unit (EIU) 
required new development and qualification testing. Antenna Type 28 was a new 
design item and subject to qualification unit and test costs. 


Another major TT&C development was the tape recorder. Initial 
development of type 31 tape recorded failed and this subcontract was terminated. 
Subsequently type 32 tape recorder was substituted. Both of these subcontracts 
are shown detailed at the bottom of Fig. 5-8, p.4. 
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Fig. 5-7 (sheet 1 of 2) Stabilization and Control Subsystem Costs 


k 






V -?K 4 V •*; 










LCPP-10 


BREAKDOWN 


S£\zst?& 


COAl27T/tx/£t 




NON-RECURRING DESIGN & DEVEL. COSTS 


LABOR ODC Total 

Direct .$ T O/lI $ |pts ?« Kfc'l J |!!"A ft Other :y| MR’ $ 



Note: $ may not add due to rounding; all $ in thousands of real year dollars. 


Fig 5-7 (sheet 2 of 2) Stabilization and Control Subsystem Costs 
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- TT&C subsystem involved two more major subcontracts for type 20 
transmitters, receiver demodulator type 4, and PCM-6 including submultiplexer 
type 3. These are itemized in Fig. 5-8, p.2. Fig. 5-8 shows the total SESP TT&C 
subsystem costs. 

- 5 . 2.7 Experiment Support Equipment 

Although SESP 71-2 experiments were GFE , the SESP program was 
responsible for their integration with the spacecraft. This effort included develop- 
ment and installation of supporting and mounting structures, mechanisms, power 
distribution, and some instrumentation. The specific equipment hardware costs 
involved were minor and are included in the particular affected subsystem. 


The overall development effort and integration analyses constitute the 
primary NR costs associated with experiment support equipment and integration 
and are shown in Fig. 5-9, (These costs do not include experiment ground support 
equipment, which was primarily GFE). 


.5.3 System Costs 

The system/program level costs were aggregated in the following cost 
categories. 


* o Program Management and Data 
o System Engineering 
. o System Test 

o System Integration and Final Assembly 
o System Reliability 
o System Quality Assurance (Q. A. ) 
o System Spacecraft Test Equipment (STE) 

o Ground Support Equipment (GSE) 

o Systems Spares 

o System Launch Operations and Support 
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Notet $ may not add due to rounding; a!! $ in thousands of real year dollars. 


p ig. 5-9 SESP 71-2 Experiment Support Equipment Costs 
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Most system level cost tables show two totals. One total represents the 
costs allocated at the system level only. The other larger total represents the 
sum of the costs at the system level, as well as those which have been previously 
allocated to the subsystems. This was done to facilitate cost impact analyses at the 
total functional level, as opposed to having some of these functions buried in the 
subsystem costs. 

.5.3.1 Program Management and Data 

The SESP 71-2 system level program management, configuration and 
data management, and data costs are shown in Fig. 5-10. 

The total system level program and configuration management cost of 
$477K excludes program management costs of $488. 4K allocated to subsystems, 
test, STE, reliability, Q. A. , GSE and systems engineering functions. Thus in 
reality $965. 4K of SESP program costs are due to program and configuration 
management. (Lower total in Fig. 5-10). 

The data costs could only be identified at the system level. These costs 
represent $107K. 

.5.3.2 System Engineering 

Fig. 5-11 presents the SESP 71-2 system engineering costs. These 
consist of requirements analyses and system level engineering analyses. No 
system engineering costs could be identified with specific subsystems, therefore 
none were allocated to subsystems. However, as shown in Fig. 5-11 there was 
system engineering effort allocated to GSE. 

Since SESP 71-2 was a high inheritance program with little design 
engineering effort, system engineering appears as a large percentage of the total 
program cost. ’ ■ ' 
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Note: $ may not add due to rounding; all $ in thousands of real year dollars. 


Fig 5-11 SESP 71-2 System Engineering Cost 
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.5.3.3 ' System Test 

The system test costs are shown in Fig. 5-12. These costs include the 
system level test costs only with the black box/major assembly qualification test 
costs allocated to appropriate subsystems as shown in the previous figures. 

These costs represent complete spacecraft functional systems test 
as well as integrated (spacecraft and experiments) system test including test support 
and test data. The GFE experiments were fully tested and accepted by the experi- 
menters prior to the integrated system test. However, due to some discrepancies 
in experiments, the integrated system test was done twice at the Sunnyvale Test 
complex C-7. Then final systems test was completed at VAFB after vehicle 
shipment. . .... 

SESP 71-2 was not required to have a system level thermal vacuum 
test. The electromagnetic compatibility testing was performed on new components 
only. These cost savings offset the repeated system test costs. Fig. 5-13 shows 
the SESP integrated systems test schedule and primary activities involved. 

.5.3. 4 System Integration and Final Assembly 

The system level integration costs, including the SESP 71-2 vehicle 
final assembly and checkout costs , are shown in Fig. 5-14. Included in the final 

I ; 

assembly cost is a cost for disassembly of an existing DTV (Development Test 
Vehicle) from another A. F. program. Parts and some structural assemblies from 
this DTV were used on the SESP 71-2 vehicle with the SESP program paying for 
the disassembly and refurbishment costs if any were required. 

Figures 5-15 and 5-16 show the final assembly and equipment installation 
schedules, respectively. Also identified are the primary tasks associated with 
physical SESP 71-2 spacecraft system integration. 
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. .The system level integration costs in -Fig. 5-14 do not include the inte- 
gration costs of $564, 5K which were allocated to subsystems, GSE , and spacecraft 
test ejquipment (STE). These are shown at the bottom of Fig. 5-14 with the system 
level integration cost added in, but excluding the physical final assembly integration 
costs. 

.5,3.5 System Reliability 

With the exception of the minor reliability costs allocated to two TT&C 
black boxes and GSE, all the reliability costs were collected at the system level 
as shown in Fig. 5-17. The reliability costs include maintainability, safety, 
production, materials and process engineering, as well as the basic reliability 
functions. 

.5.3.6 System Quality Assurance 

The total SESP 71-2 quality assurance program cost of $1000. 7K is 
shown in Fig. 5-18. Included in that total are costs allocated to subsystems and 
extensive system test support. If these allocated items are deducted, the remain- 
ing system level Q. A, cost, is $309. 6K as shown at the bottom of Fig. 5-18. The 
G. A.- functions identified are as shown in the figure. Also shown are the Q. A. 
documentation and report costs which could be isolated. 

.5.3.7 Spacecraft Test Equipment 

The spacecraft test equipment (STE) costs were collected at the sub- 
system (black box) level and at the total system level. The system level STE 
costs consisted of modifying the existing Sunnyvale C-7 test complex STE, 
experiment STE, and overall STE integration and support. Fig. 5-19 shows the 
SESP 71-2 system test AGE (referred to in this report as STE) activities and 
schedule. 

Fig. 5-20 presents the SESP 71-2 System' level STE costs of $595. 2K. 

If the STE costs allocated to subsystems ($141. 9K) are included, the total SESP 
program STE costs were $737, 0 K. The STE allocated to subsystems , mostly at 
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Fig. 5-17 - SESP 71-2 System Reliability Costs 
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Fig 5-18 - SESP 71-2 System Quality Assurance Costs 
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Fig 5-19 - SESP 71-2 System Test AGE (C-7) 
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Fig 5-20 (sheet 1 of 2) SESP 71-2 Spacecraft Test Equipment Costs 


i 


*i * ( . i j 


















































































sussYsrz*) STS’ 


r2> sossy. 



Note: $ may not add due to rounding; all $ in thousands of real year dollars. 


Fig 5-2G (sheet 2 of 2) SESP 71-2 Spacecraft Test Equipment Costs 
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Fig 5-21 - SESP 71-2 Ground Support Equipment Costs 
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Note: i may not add due to rounding; all $ in thousands of real year dollars. 


Fig'. 5-23 - SESP 71-2 System Spares Costs 
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.The SESP 71-2 launch base activities .and schedule are indicated in 
Fig. 5-25. LMSC responsibilities as the SESP 71-2 integrating contractor were 

limited to providing operational technical assistance and support services to launch 

i i 

activities performed by others. The Thorad booster was the responsibility of 
McDonnell Douglas. LMSC/VAFB had the launch services contract. Aerospace 
Test Wing 6595 had the launch responsibility and A. F. Satellite Control Facility 
provided tracking, telemetry, and command services. 

.6.0 SESP 71-2 COST PERCENTAGE DISTRIBUTIONS 

The iSESP 71-2 program costs as allocated are summarized in Fig. 6-1. 

Also shown are percentage distributions derived in terms of manhours , as well as 
dollars for the total program costs. Fig. 6-2 shows these costs in terms of basic 
accounting cost categories and their percentage distributions. 

Utilizing the total SESP 71-2 costs for some of the selected functions, 
including the portions which have been allocated to subsystems, Fig. 6-3 was 
prepared. It attempts to show the maximum cost and percentage of program cost 
for a given function if all the buried costs are summarized under a given function. 
Only functions thought to be of special interest are shown and there are cost over- 
laps . 

- * For example maximum quality assurance cost of $1000. 7K or 8.3% max. 

of program cost includes Q. A. system test support of $200. 6K charged to system 
test.. (See Fig. 5-18). The maximum cost of $937. IK for test function shown 
in Fig. 6-3 includes the above Q. A. support and test costs allocated to subsystems, 
in addition to the system test costs shown in Fig. 5-12. 

Another function of interest is system integration. Considering only the 
"software" costs, excluding the physical hardware final assembly costs, the maxi- 
mum integration costs identified in the SESP program including subsystems charges 
were $699K (See Fig. 5-14) or 5.8% of program cost and 11. 6% of program in-plant 
manhours. Combining "software" system integration function with system engineer- 
ing function, the maximum estimated cost to SESP becomes 17.3% of program cost 
and 21. 3% of in-plant manhours. 
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Fig. 6-1 - SESP 71-2 Program Cost Summary 
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Fig. 6-2 SESP 71-2 - Basic Cost Category Distribution 
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Selected Functions 

Max. % of Total Program 

*• • 

Hours 

Cost ($X1000) 

Quality Assurance 

61397 11.6 

1000.7 8.3 

Test 
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Syst. Integration 
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17.3 
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327.1 2.7 

Mfg. Plan & Tooling 
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313.3 2.6 

Support Hdw. 



STE 

36130 6.8 

737.0 6.1 

GSE 

14306 2J_ 

325 .6 2/7 

Combined 
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8.8 

Spares 
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196.1 176 



LOCKHEED MISSILES & SPACE COMPANY, INC 



LCPP-10 


The costs identified with spacecraft test equipment (STE),both at the system 
and subsystem level, represent 6. 1% maximum of SESP program cost. Including 
GSE with STE costs results in 8.8 maximum percentage of program cost as shown 
in Fig. 6-3. 

These percentage figures are considerably higher than the percentages 
shown for these functions in Fig. 6-1, where subsystems were allocated a 
substantial portion of costs for some functions. From the standpoint of program 
to program comparisons, care has to be taken in extracting total costs for a given 
function or comparable allocated cost share. 
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Lockheed Missiles and Space Company is currently performing the Low-Cost Program 
Practices study for NASA Headquarters under contract No. NAS W-2752. The principal 
objective of the study is to identify those NASA space program practices that 
contribute significantly to program costs. The Air Force Space Experiments 
Support Program 71-2 (SESP 71-2) has been identified as a low-cost program. The 
SESP 71-2 program costs have, therefore, been analyzed to determine the impact 
of various program practices on the costs of a typical unmanned, automated space- 
craft program. The results of the analyses are presented in this engineering 
memorandum. 
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SESP 71-2 PROGRAM PRACTICES ANALYSTS 

This discussion of low-cost and high-cost program practices identified from 
the SESP 71-2 program history is subdivided into three parts to facilitate 
analysis. The three parts consist of: 1 ) inheritance, 2) practices besides 

inheritance, impacting the non-recurring cost and 3 ) practices impacting the 
recurring unit cost. The recurring operations practices were not studied. 


H.l INHERITANCE 

Exploitation of inheritance from prior programs constitutes a major low-cost 
practice resulting in significant program cost savings or more programs/benefits 
the same amount of dollars. Spacecraft program inheritance varies depend- 
ing on: 1) the degree of off-the-shelf design applied; 2) amount of flight 

qualified hardware used; 3) the extent of existing software, test equipment, 
ground support equipment.; and in some instances, 4) existence of facilities. 

Use of inheritance has been implemented as a practice in many spacecraft pro- 
grams, therefore it is not a new practice. However, the amount of cost savings 

achieved with this practice is highly dependent on the degree to which inheri- 
tance is implemented. 

In the case of SESP 71-2, the degree of inheritance was very high. Fig. 1-1 
shows the SESP spacecraft major assembly and black box status in terms of . 
existing design and program peculiar design (modified or new equipment). From 
this approximate measure, 6 3 ? 0 of the spacecraft was of existing design 24 * 
was modified equipment, and 13?* was new. These figures are somewhat misleading 
since some of the assemblies such as structures may have been of a new design ’ 
but were not; functionally critical to the mission and, therefore, did not 
represent the same cost implications as new active critical components. The 
tter require considerably higher investment in development, qualification 
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hardware, and testing. The last column in Fig. 1-1 lists the six new mission 
critical, items which had not been flown previously. This 5$ of equipment 
constituted the major SESP 71-2 development and test hardware cost. These 
items were: pulse code modulation unit type 5 (PCM-5), experiment interface 


unit type 1 (EIU-l), antenna type 28, heat exchanger coolant system, and two 
items supporting the experiments (RTD-806 array release mechanism and shaft 
position indicator). The modified mission critical equipment development 
(7$ of the spacecraft boxes/assemblies) was concentrated in the tape recorder 
and other TT&C black boxes and the electrical power subsystem boxes. 


Contributing to the extensive SESP 71-2 spacecraft hardware inheritance was 
the high support equipment inheritance represented by existing GSE and test 
equipment. An approximate count of support equipment shows, 23$ were new 
with more than half of these being in the ground handling equipment category 
and therefore relatively inexpensive. The existing equipment comprised about 
45$, the modified equipment about 29$, and GFE 3$ of the items. Most of the 
expensive launch complex GSE was existing, and the test equipment complex 
required primarily minor mo.difications . 

Software can be a considerable development cost driver. On the SESP 71-2 
program, software was existing (only $12.K was spent on it). 


In- summary, the impact of inheritance as a low-cost practice implemented in 
the SESP 71-2 program is shown in Figure 1-2. It is difficult to estimate 
precisely the dollar savings contribution of inheritance. This figure shows 
the results of ROM parametric cost estimates derived for a spacecraft with 
SESP 71-2 performance and mission characteristics. Considering the case with 
no inheritance, development "from scratch," $55M and $36 m non-recurring 
development costs were estimated for the conventional development approach 
and the low-cost approach, respectively. The parametric estimates for the 
high inheritance case resulted in $9M to $6 m non-recurring cost, allowing 
an estimated cost savings due to inheritance in the $46 to $30 million range 
depending upon the development approach. 


5 


LOCKHEED MISSILES & SPACE COMPANY. INC. 


CHARACTERISTICS : 

o OFF-THE-SHELF DESIGN 
o USE OF FLIGHT QUALIFIED HARDWARE 
o EXISTING SOFTWARE, TEST EQUIPMENT & GSE 


SESP 71-2 PROGRAM : 

o ONLY SIX NEW ITEMS {%) NOT FLOWN PREVIOUSLY 
o MIN. DEVEL. & QUAL. TEST HARDWARE - COST «= $350K 
o USE OF EXISTING STE & GSE - COST ^ $900K 


ESTIMATED COST IMPACT OF SESP INHERITANCE : 

CONVENTIONAL 

COSTS APPROACH 


IOW COST 
APPROACH 


DEVELOPMENT "FROM SCRATCH" $55M 

DEVELOPMENT WITH HIGH 9 

INHERITANCE 

INHERITANCE SAVINGS $46M 


$36m 
' 6 

$3QM 


SESP 71-2 ACTUAL NR COST 


$9 . 66m 


DIFFERENCE DUE 
TO LOW COST PRACTICES 

$19M 

$ 3M 


Fig. 1-2 


Mag or Low Cost Practice: Inheritance 
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The SESP 71-2 actual non-recurring development cost was $5.66m, which is 
close to- the low-cost approach estimate of $6 m and over $ 3 M lower than the 
conventional approach estimate with high inheritance. Since SESP 71-2 was 
built of conventional spacecraft hardware, it did implement low-cost prac- 
tices ^beyond inheritance to reduce its non-recurring cost below the expected 
high inheritance, conventional spacecraft level. 

tl ' 2 lff sp 71-2 PRA CTICES WHICH PRIMARILY IMPACTED THE NON- RECURRING COST 

The major practices, as identified from the SESP 71-2 history and financial 
records, which impacted primarily the SESP 71-2 non-recurring cost are listed 
in Fig, 2-1. Since, in many cases, the historical records do not provide the 
actual costs at the level of detail required, estimates had to be generated 
to provide some quantification of the practice. 

- The following discussion describes each practice in detail, both the high and 
the low-cost, and the means of estimating where actuals were not available. 

H' 2 - 1 ' Low-Cost Practice: Overdesigned Structure 

Impact: SESP 71-2 NR Cost Savings Estimated at $120K 

Description : 

Overdesign of the structure permitted elimination of the qualification test, 
and spacecraft bending moment and loads analyses were not required. The 
estimated cost savings of $70K for the testing and $50K for the analyses are 
based on analogous activities on other programs. 

Conclusion : 

Structures overdesigned by margins of several factors of safety do not require 
detailed strength testing and subsequent analyses. Such design and test 
practices for structures have been advocated by the previous LMSC Low-Cost 

Payload Studies as a part of low-cost design approach. 
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LOW-COST PRACTICES 


DESIGN & TEST: 


o STRUCTURE OVERDESIGNED, NOT REQUIRING: 
QUALIFICATION TEST 
S/C BENDING MOMENT & LOADS ANALYSES 
o SYSTEM THERMAL VACUUM TEST NOT REQUIRED 
o EMC TEST OF NEW COMPONENTS ONLY 


SUBTOTAL 


DOCUMENTATION: 


o MLL-D-1000 FORM 3 DRAWINGS 

» 

o USE OF EXISTING PROCEDURES, SYSTEMS & DOCUMENTATION 
o DELETION OF TWO CDRL ITEMS 


SUBTOTAL 


PROGRAMMATIC/MANAGEMENT : 


o SHORT (11 MOS.) DEVELOPMENT SPAN (EST. 5 MOS. SAVINGS OF 
ENG. & PROG. SUPPORT AT $300K/M0.) 

o MINIMUM IN SCOPE CHANGES (CONSIDERABLY LESS THAN AVERAGE 
NASA FOLLOW-ON S/c) 

' SUBTOTAL 


PIIGH COST PRACTICES 


o FIRST TAPE RECORDER SUBCONTRACTOR COULD NOT MEET SPECS 
o LATE GFE EXPERIMENT SPECS CAUSED P/L MODULE REDESIGN 
o STRINGENT CONTAMINATION CONTROL REQUIREMENTS 

SUBTOTAL 

NET LOW-COST PRACTICES NR SAVINGS 


Fig, 2-1 Practices Impacting SESP NR Cost 








EST. SAVINGS 

(70-71$) 


$ 70K 
50K 
600K 

N.A. 

$720K 


$4ook 

200K 

10K 

$6l0K 

$1500K 

500K 

$2000K 


-$174k 

- 90K 

- 65K 
-$329K 
$3000K 
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II. 2.2 Low-Cost Practice: Eliminate System Thermal Vacuum Test 

Impact : SESP 71-2 MR Cost Savings of $6 o 8K 

Description : 

System thermal vacuum test initially considered as required was eliminated at 
a cost savings of $6 o 8 k (as quoted to the customer). 

Conclusion : 

The above savings, as well as the other test savings shown in Fig. 2-1, 
resulted in about $720K cost reduction which represents almost 5^ savings 
of the hypothetical program cost. The program cost savings are discussed 
in Section .4. 

lb 2.3 Low-Cost Practice: Reduce and Simplify Documentation Requirements 

Impact : SESP 71-2 Estimated MR Cost Savings of $6lOK ' 

Description : ... 

SESP 71-2 utilized MIL-D-1000 Form 3 drawings which represent 43^ savings 

over the Form 2 ^ commonly applied in LMSC AF programs. Based on a nominal 

(n) 

program Form 2 drawing cost of $2.24 m v 1 of which 63$ are assumed to be 
existing drawings (Fig. l-l), about $400K savings can be derived from Form 3 
application. ($2. 24M + $1.72M simple changes x .37 x .43 x 16/22 « $400K. 

The l6 a 22 is the inflation adjustment to 1970-71 SESP 71-2 manhour rates.) 

The use of existing procedures, systems, and engineering documentation (speci- 
fications, etc.) was estimated to save $200K. The $10K savings resulted from 
deletion of two specific CDRL items . 

References : (l) PER Handbook, Appendix 1 to Final Report Low Cost Program 

Practices for Future NASA Space Programs, LMSC-D387518, 

30 May 1974, p. 3-29. 

(2) Op. cit., p, 3-25. 
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Conclusion : - 

Documentation costs are extensive and' well hidden in the other costs of the 
program, primarily labor. Thus reduction of the papermill and simplification 
such as Form 3 vs. Form 2 drawings brings substantial savings. In the case 

of SESP 71-2 the estimated $ 6 lOK savings represented about k% program cost 
savings (see Fig. 4-1). 


tl- 2 - 1 * Now- Cost Programmatic and Management Practices 
Impact : High. For SESP 71-2 this impact was estimated very ROM to be $2M 


Description : 

The two practices with high cost impact considered under the programmatic/ 

management practice areas were the development time span and the number of 
• in-scope changes. 

The development time span has a significant impact on costs, because the 
longer a team is on a project the higher the labor costs. The development 
span for SESP 71-2 was 11 month's maximum (the final design review) and could 
be considered as short as 8 months at which time engineering was 7 % complete 
and final assembly had been started. The total time span from go-ahead to 
launch .was 18.5 months, which is about 9 months shorter than some MSA pro- 
grams with medium to high inheritance. For example: 


OSO-F 

18 months till delivery (launch, 12 + mos. later 

NIMBUS -E 

, due "to late experiements ) 

31 months till launch 

MM' 71 

31 months till launch 

MVM'73 

29 months till launch / 

Pioneer F 

29 months till launch 

Average 

27.5 months till launch 
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The contribution of the SESP 71-2 short development span to savings of NR 
cost was. estimated to be about $1 . 5M on the basis of an assumed additional 
5 months of effort (-27.5-7-18.5 x 11 mos.) at the average SESP 71-2 rate of 
expenditure of $300K per month for engineering and program support efforts. 

The cost impact of in-scope changes has been identified in a previous study 
(see Reference 1.) as contributing significantly to program cost growth. In 
the SESP case the low number of in-scope changes contributed about 14$ to 
cost growth, which is considerably less than NASA follow-ow spacecraft con- 
tracts averaging 25$. 

The resultant cost savings are difficult to isolate since there exists an 
overlap with degree of inheritance and development time span. The arithmetic 
computation of savings was reduced by half to provide a conservative ROM 
estimate of $500K as savings attributed to small number of in-scope changes. 

Conclusion ; 

Development time span and i'n-scope changes are very .significant cost driving 
practices, and should be given -thorough analysis in the program planning 
stages. 

II. 2.5 High Cost Practices Impacting SESP 71-2 NR Cost 

Impact ; The three specific practices identified caused dissavings of about 
$330Kv 

Description :. 

The three practices costly to the SESP 71-2 development were: 

l) The original tape recorder subcontractor could not meet the 
design specifications and the subcontract was terminated at 
a $174K cost to the program. 
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2) Late definition of one of the GFE experiment specifications 

' arid the related requirements- imposed on the sp,..ecraft caused 
major payload module redesign effort at an estimated cost of 
$90K. 

3) Very stringent contamination control requirements imposed 
by one of the GFE experiments were estimated to add about 
$6jK to the cost of the program. 


Conclusions : 

These three practices caused dissavings of over 2 i of the hypothetical 
program cost as estimated in Fig. 4-1. 


II *3 PRACTICES IMPACTING SESP 71-2 UETT COST 

. The ^practices impacting SESP 71-2 unit cost are summarised in Fig. 3 -i. 
Again both low-cost and high-cost practices were identified. These are 
discussed and quantified individually in the following sections. 

I 1 ’ 3 - 1 bow-Cost Practice:. Existing Equipment Reuse 


Impact : SESP 71-2 Unit Cost Savings of $371K 

Description : 

Reuse of existing Agena hardware available from other A.F. and MSA programs 
in the form of a DOT, black-box test or spares hardware, and common equipment 
inventory. This equipment became GR5 to the SESP program, with the program 

paying for equipment refurbishment if any was required. The cost savings to 
the program were $371K and 5381 manhours . 
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| LOW COST PRACTICES: ■ EST. SAVINGS 

" . (70 - 7U) 

o EXISTING EQUIPMENT REUSE (REFURB. OR GFE) s * 37 OK . 

o MINIMUM SPARES (TAPE RECORDER, 3 BATTERIES & PIECE PARTS) 400 

o TEST HARDWARE UPGRADING TO FLIGHT HARDWARE 100 

o QUANTITY BUY OF EQUIPMENT 30 

SUBTOTAL ’ $ 91QK 


HIGH COST PRACTICES : 

o EXTENSIVE SYSTEM TEST (LATE GFE EXPERIMENTS & REPEATED TEST) $-240K 



ij 


Fig. 3-1 Practices Impacting SESP Unit Cost 
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REUSED & GFE 

EQUIPMENT - MAJOR ITEMS BY 

SUBSYSTEM: 

Est. New 
Cost 

Savings 

Subsystem 

Equipment 

Cost to 
SESP 

S & C 

Guidance Module 

$123K 

$350K 



Flight Controls 

46 

75 



ACS N 2 Tank & Plumbing 

- 

16 



N 2 Regulator 

- 

9 



Thrust Valve Cluster 

13 

18 


■ 

Total 

$182K 

$468k 

$ 286 k 

TT&C 

Antenna Type 7 

_ 

2 



Magnetic Timers 

- 

3 



RF Switches 

- 

1 



Total 

- 

$ 6k 

$ 6k 

Electrical 

Power Distribution J-Box 

5 

ll 



Fwd. Safe/Arm Box 

1 

2 

' 


Inverter 

- 

17 



Main Pwr. Tr. Switch 

- 

4 



Amp. Hr. Meters 

- 

2 



DC/DC Converter 

- 

13 



Solar Panels & Substrate 

8 

20 



Total 

$ 14k 

$ 69 K 

$ 55K 

Propulsion 

Isolation Valves 

$ 13 

$ 29 - 


. 

He Tank 

1 

8 



Bellows & Misc. 

l 

2 



Total 

$ 15 K 

$ 39K 

$ 24k 

Total Equipment Savings including GFE 



$371K 


Conclusion : 

Equipment reuse contributed total savings of 24 $ to SESP program cost. How- 
ever, a portion of these savings was due to reuse of equipment as GFE items,. 
The equipment reuse practice should he applied when commonality of hardware 
and inventory of such equipment exists. On the average, the SESP 71-2 pro- 
gram paid refurbishment costs of 30 $ or less on the equipment requiring 
refurbishment . 
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11.3.2 Low-Cost Practice: Minimum Spares 

Impact:. SESP 71-2 Recurring Cost Sayings Estimate of $403K. 


Description : 

Minimum spares procurement and make costs consisted primarily of the costs 
of one tape recorder, some batteries, and parts only for critical black boxes 
For other critical equipment items, arrangements were made with on-going A.F. 
programs to obtain spare items on short notice if they were required at the 
la/unch base. 


SPARES BREAKDOWN: 


Subsystem 


Cost to Est . Nominal 


Equipment 

SESP 

Cost 

Savings 

S&C 

Horizon Sensor 
Gyro 

Pro-Rated Spares Maint.* 

10.0 

$ 30. OK 
6.5 
5.8 


• 

Total 

$ 10. OK 

$ 42. 3K 

$ 32.3 

TT&C ' 

Tape Recorder 
PCM- 5 1 

$122.7 

$122.7 

135.6 


- 

EITJ L 

FCM-3 f 

1.6 (Parts) 

55.8 

23.4 



Pro-Rated Spares Maint.* 

10.0 

51.9 

- 

Electrical 

Total 

Fuse Resistor J-Box 
Change Controller 

$134. 3K 

$389. 4k 

8.0 

33.0 

33.0 

14.0 

13.0 
9.5 

$255.1 


Power Cont. Logic Assy. 
Shunt Regulator 
ONR Control J-Box 
Battery Type 30 (l) 

6.4(Parts) 



Battery Type 7 (3) 
Pro-Rated Spares Maint.* 

27.9 

10.0 

27.9 

22.0 



Total 

$ 44. 3K 

$ 160 . 4K ^ 

$116. IK 

Total Spares Costs & Savings 

$188. 6 K** 

$592. IK 

$403.5K 

Spares of Flight Unit Cost: 

3.2 i 

lO.lfo 

6.9^ 
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NOTES : 

* Spares Maintenance and Repair costs on SESP 71-2 represent 15.9* of the 
spares hardware. Thus as spares hardware goes up these maintenance costs 
go up. These costs were pro-rated to subsystems. 

**This cost represents the spares for the flight vehicle excluding $7.5K of 
GSE spares . f ‘ 

Conclusion : 

The minimum sparing practice on SESP 71-2 (3.2 of unit cost) saved about 
3 % of the program cost as compared to the nominal sparing practices repre- 
sented by 10 - 20 % of flight unit or complete spare vehicles. In the case 
of SESP 71-2 this minimum sparing was possible due to hardware commonality 
and common equipment inventory with other programs. For example, the Type 
30 battery spare was a non-flight item as declared by another program due to 
critical out of tolerance dimension. This condition was not critical to SESP 
and electrically the battery met all specifications. 

H-3-3 lo w Cost Practice: Refurbishment and Upgrading of Test Hardware to 

- Flight Status 

Impact : SESP 71-2 Unit Cost Savings of $110K 

Description : 

This practice consists of reusing the developmental and qualification test 
hardware, after refurbishment and upgrading, as flight hardware. On the 
SESP 71-2 the major occurrences of this practice involved the refurbishment 
of the heat exchanger qualification unit into a flight unit at estimated 
savings of $90K, and reuse of major structural elements such as forgings at 
savings of $20K. The latter were obtained as GFE items by disassembling a 
DTV (Developmental Test Vehicle) from another Agena program. 
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Conclusion ; 

This practice contributed minor savings (less than 1 ( j 0 ) to the SESP 71-2 
program cost, since this program had very little test hardware to begin with 
However, on programs with considerable test hardware, the conversion of DTV' 
and qualification units into flight hardware may save considerably more and 
be a significant low-cost practice. The refurbishment of the $90K heat 
exchanger qualification unit to bring it up to flight status cost $lhK or 
16 $ refurbishment rate-. 

1**3 - L Low-Cost Practice; Quantity Buy of Equipment 

Impact: Variable Savings Depending on the Cost of the Items and No. 

of Items Involved 

Description ; 

Quantity-buy savings on common equipment items for several programs may be 
substantial. This practice, is implemented as much as possible by contrac- 
tors central procurement organizations, which order a given equipment item 
in quantity to meet the requirements of several programs. As an extension 
of this practice, LMSC has procured items in quantity for "common equipment - 
inventory" under a specific contract provision from the A.F. This equipment 
inventory would then be available to the using programs at lower costs than 
the costs the individual program would have to incur if it procured to 
satisfy only its own requirements. In some cases the common inventory equip- 
ment was treated as GFE material. 

In the case of SESP 71-2, a single flight vehicle, the significant quantity- 
buy practice involved the Bell engine. It was ordered in a 14 engine lot at 
an average price of $359K. ; The price for a single engine buy would have 

been considerably higher.) However, due to schedule pressures, the SESP 
program required an engine earlier than the scheduled delivery date and was 
able to substitute an engine from another program at the previous procurement 
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price of $330K. This resulted in additional savings of $29K plus a further 
$3K savings due to the fact that the substitute engine' had already been 
screened (engine reliability verification) by the other program. 

Other SESP 71-2 items benefiting from quantity procurement were batteries, 
switches, and many other parts for which it is difficult to extract the 
amount of cost savings involved. 


Conclusion : 

Only small savings could be identified and attributed to the practice of 
quantity buying on the SESP 71-2 program, but as a cost-saving practice it 
should be implemented whenever conditions permit. 

1 1.3. 5 High Cost Practice: Extensive System Test 

Impact: SESP 71-2 Unit Cost Dissavings of ft* $460K 

Description : 

The system test budget was overrun due to the following extensive testing: 
l) integrated system test utilizing experiment simulators, 2) integrated 
system test repeated with installed experiments, 3) final system test with 
all specifications and requirements imposed, and 4) final systems test com- 
pleted at VAFB. 

The delays and repeated testing were due to late GFE experiments, non- 
availability of the remote site for testing of one experiment, and problems 
with the overall vehicle schedule slip of three weeks which caused scheduling 
problems at the system test complex. 

Associated with higher test hours were the hour increases in data processing/ 
analysis,- documentation, and test management, as well as engineering and Q.A. 
support, 
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Conclusion : 

These test dissavings represent about'2<g, of the program cost and can be elim- 
inated by improved planning and scheduling. If the associated support cost 
impact is considered, the total dissavings approximate 3* of program cost 
with an additional estimated $220K expended in systems engineering, a. A., 
management, and documentation activities. 

|| 3^6 High Cost Practice: In-Hous e Make of PCM-5/EIU 

Impact : SESP 71-2 Unit Cost Dissavings of a $200K 


Description : 

The decision to make in house the new Pulse Code Modulation Unit Type 5 
(PCM-5) and the associated Experiment Interface Unit (EIU) resulted m cos 
increase of $200K over the subcontractor's proposal. Contributing to .. 

'this overrun were the PCM-5 crystal oscillator failure, which required requal 
ification and associated support activity. It is impossible to say whether 
the subcontractor would have had the same overrun. 

Conclusion : 

More careful evaluation of in-house make vs. subcontracting, and the type of 
contract arrangement with the subcontractor might have reduced this negative 
cost impact of about 1* of SESP 71-2 program cost. 

II .4 SESP 71-2 PROGRAM PRACTICES SUMMARY 

The absolute estimated dollar savings and increases of SESP 71-2 practices 
were converted to percentages of program cost to provide a more generalised 
view of the low and high-cost practices impact on the program. 

In order to permit percentage computations a hypothetical program cost had 
to be derived, omitting the inheritance impact. Pig. 4-1 shows this deriva- 
tion and the contributions of the various practices to the program, The 
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SESP 71-2 ACTUAL PROGRAM COST 
LESS HIGH-COST PRACTICES: 
o HR PRACTICES 

o EXTEHSIVE SYSTEM TEST & SUPPORT 
o IN-HOUSE MAKE VS. BUY 


PLUS LOW-COST PRACTICES: 
o DESIGN & TEST 
o DOCUMENTATION 
o PROGRAMMATIC/MANAGEMENT 
o EXISTING EQUIPMENT REUSE 
o MINIMUM SPARES 
o OTHER UNIT COST PRACTICES 


SESP 71-2 HYPOTHETICAL PROGRAM COST 


MET LOW COST PROGRAM PRACTICES SAVINGS 


$12. 026M 


- 0.329 

- 0.460 

- 0.200 


+ 0.720 
+ 0.610 
+ 2.000 
+ 0.370 
+ 0.400 
+ o.i4o 


$ 19 . 277 M 


$ 3.251M 


(INHERITANCE SAVINGS OF $30 - $46m NOT CONSIDERED) 


Fig. 4-1 • Program Practices Cost Impact 


% OF HYPOTHETICAL 
PROGRAM COST 


18. 7% 


2 . 2 % 

3.0 

1.3 


+ 4.7 
+ 4.0 
+13.1 
+ 2.4 
+ 2.6 
+ 0.9 


100 . 0 % 


21 . 
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total net sayings due to SESP 71-2 low-cost program practices were estimated 
at more than 20$ of the hypothetical program cost. In other words, the 
actual SESP 71-2 cost represented 79$ of the cost which could have been 
incurred if the low-cost practices were not implemented. 

The practice of inheritance was not considered pertinent in this evaluation, 
because it may have been the major driver which permitted the SESP 71-2 to 
become a program in the first place. 
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PROGRAM EFFECTS RELATIONSHIPS (PER) SURVEY 


12.1 Background 

Program Effects Relationships (PER's) are line graphs, tar charts, data 
tables, etc. which identify the key parameters of alternative practices, 
management and technical, within a general program practice area and relate 
costs 'for accomplishing each alternative practice. For example, there are 
several well defined levels of Quality Assurance ((Commercial, JANTX (general 
military), Hi-Rel military and Super Hi-Rel Lockheed/military^ . Each has 
separate sets of requirements and conditions to he met which in turn result 
in costs which range from one for commercial QA programs to eight ti‘mt=>s as 
much 'for Super Hi-Rel QA programs. Costs of individual components which do 
exactly the same job in a system but which are qualified to the requirements 
for different QA levels can range from one for the commercial level to l6 
oi\l8 t times that amount when qualified in accordance with Super Hi-Rel 
requirements. In concept PERs differ greatly from Cost Estimating Re- 
lationships (CERs). CERs, relate the cost of hardware/ software elements ■ 
to physical parameters of these elements. Use of historical CER’s to pre- 
dict cost of future programs assumes that future elements of hardware/soft- 
ware will always be developed in the same. way and in accordance the same 
practices as were used in historical programs. Dependence on the use of 
such traditional CERs for estimating cost of future programs tends to 
preclude the use of new innovative management and technical practices which 
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could become low-cost drivers in future programs. In addition, when costs 
predicted via the traditional CEE route are adjusted for inflation/escalation 
the resulting cost predictions tend to go only one way, up. On the other 
hand, the PER concept is dedicated to the proposition that new alternative 
practices can be devised which, in turn, will allow NASA to buy more spare 
operations for each NASA dollar. This concept recognizes the fact that no 
single set of practices can always be the most cost effect . set for all future 
programs, and requires PERs to be developed to show the relative costs for 
implementing any of the several alternative practices considered practical 
within a particular practice area. Thus, if and when meaningful PERs can 
be developed to cover most program practice areas they can become powerful 
tools to assist cost-effective decision making in future programs. 

During a previous study LMSC developed an initial set. of PERs and a pre- 
liminary PER Handbook based on data from historical space program records. 
NASA/LM3C considered this handbook to be a good start toward development 
of a highly desirable program planning tool. However, several ifs were 
also, recognized in the previous study effort. These ifs included but were 
not limited to; the availability of historical programatic and cost data 
to develop PERs covering the full range of program practices, PER content 
and format preferred by potential users and the probability of wide-spread 
acceptance/use of such a tool. Thus, a PER Survey was planned to be accom- 
plished during the present study (before more PERs were generated) to 
obtain potential user inputs relative the recognized ifs. 

While planning the PER Survey and developing a mutually agreed upon ques- 
tionnaire NASA/LMSC decided to offer anonymity to prospective participants 
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for the following reasons: 

a. NASA's major role has changed from that of transforming science 
fiction type space concepts into real-world space operations 
for the first time to that of providing more economical space 
operations based on experience gained via 17 years of successful 
operations and the expenditure of more than fifty billion dollars. 

b. In view of present economic conditions NASA has a more urgent need 
to acquire more space for each NASA dollar than it has had in 

all its previous existence. 

c. Candid recognition, isolation and quantification of historical 
high-cost diriving practices are essential steps in developing 
meaningful PERs 

d. The act of identifying and quantifying historical high-cost 
driving practices would place prospective participants in the 
awkward position of having to be critical (from the economic point 
of view) of some NASA practices that were used to make space 
flight a reality in the first place 

. ■ t 

e. Involved people within the NASA and industry are sensitive to 
making/accepting statements which tend in any way to be critical 
of past accomplishments and/ or practices . 

During the survey 33$ of the participants elected this option, 49$ did not 
and 18 $ did not state a preference. 

12. 2 Per Survey Objectives 

General objectives were to obtain and incorporate USER INPUTS from a repre- 
sentative cross section of potential NASA/industry users into NASA's 
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PER Handbook development efforts and to promote open dialog relative to 
developing low-cost space programs in' the near future. Detailed objectives 
included : 

a. Specific Practice Areas for which PER's are needed. 

b. Scope of Coverage needed for best user application, 

c. Practical Range of high-to-low cost practices to cover in each 
PER set pertaining to program practice areas such as: 

o Program Management o General Contracting 

o Specifications o SE&I o GSE o QA & Reliability 

d. Preferred PER and PER Handbook 'Format (s) 

e . Potential usefulness evaluation 

f. Status and availability of historical data for generating a 
full set of PERs. 

g. "How-Best-To" comments: 

o Promote implementation of low cost spacecraft programs in 
near future . 

o Apply and use PER Handbook effectively. 

» 

12.3 • Scope of Survey 

The scope of effort planned and accomplished is outlined below. 

INTERVIEWS 

o Ten MSA Field Center project/spacecraft managers, 
o Ten MSA Field Center functional division managers. 

0 Tk ree MSA/HQ, program managers (OSS," QA, OMSF). 

0 Nine Industry program/f unctional division managers representing 
at least five companies. 
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DOCUMENT INTERVIEW RESULTS 

o Answers to questionnaires, 
o Interview comments (Pro and Con), 
o Related activity/interface ideas and comments, 
o Unanticipated inputs/constraints, 
o Special data for new PER's (Specs, SE&I, GSE) 

PERFORM COMPOSITE ANALYSIS OF INTERVIEW RESULTS 

o Summarize requirements; preferences; limitations, 
o Generate "preferred" content/format ( s) for PER's and 
PER Handbook. 

o Document rationale for "preferred" PER approaches. 
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12 . 4 Participants 

Due to the anonymity^ option elected by 33 percent of survey participants 
individual names are not listed. The names of participating organizations 
and numbers of questionnaires completed by each are: 

NASA INDUSTRY 


Ames 

5 

Hughes 

2* 

GSFC 

6 

JPL 

6 

JSC 

9 

LMSC 

2 

MSFC 

7 

Martin 

3 

NASA HQ 

2 

Philco 

3 

Total 

29 

SCD 

1** 



TRW 

3 


Total 20 

* Selected teams of personnel completed two questionnaires. 

** Only one requested from Systems Development Corporation. 

Several other MSA and industry organization initially contacted were will- 
ing to participate but were not needed to satisfy the scope of effort, 
planned and, reluctantly, were not Included due to time and money constraints 

12 . 5 Survey Implementation 
Essential steps used to implement this survey were: 

a. Prepared and obtained NASA approval of a basic survey Implementa- 
tion Plan, Orientation Brochure and Survey Questionnaire. 

b. Mailed orientation brochures and questionnaires to participants 
prior to conducting face-to-face orientation briefings and 
interviews . 
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c. Provided participants the choice of 'completing questionnaires 
during interviews or to use 'the interview periods for expanded 
discussions and to mail in completed questionnaires later. 

d. Maintained follow-up contact via telephone to answer additional 
questions and to receive modified/updated inputs. 

f. At the request of several participants, supplemental agreements 
were made during meetings/interviews which allowed participants 
to: 

o Omit responses to questions he felt- unqualified 
to answer 

o Alter the wording of questions where he felt it 
would help clarify the full meaning of his response 
o Add additional questions and answers if he felt any 
significant questions were omitted. 

12 .6 -Survey Results 

Results are presented in five separate parts. These are: 
o Compiled on a questionnaire form 
o Summary of key responses 
o Additional comments from oral discussions 
o Conclusions 
o Recommendations 

12 . 6 . 1 Results Compiled on Questionnaire 
To the extent practical all relevant questionnaire response are displayed/ 
quantified on a copy of the questionnaire form used in the survey. This 
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"long-form" format is used to provide the reader the best possible feel 
for the "full significance of responses obtained including why some resulting 
answers tend to be critical of a few historical practices (NASA's, DOD's and 
Industry's) from the cost point of view. It was also used to provide the 
reader an opportunity to compare his feelings, beliefs and convictions with 
those obtained in this survey. 

Quantifible results have been quantified to reflect the total number of NASA 
and industry responses to each question separately and combined. No attempt 
was made to quantify differences In response trends between program managers 
and functional managers nor between organizational levels of individual 
participants. Such breakdowns were considered meaningless due to the facts 
that some program managers are ex-functional managers (visa versa) and there 
is no common denominator for comparing responses from individuals in organiza' 
tiona^. levels ranging from 'first -line managers to Directors in the NASA and 
the president of one industrial company. 

A total of 49 participants completed or supervised teams to complete 
questionnaires to the extent they deemed practical. Of these, 15 are 
program/project managers, 33 are functional managers, 1 is president of an 
industrial company, 29 are NASA personnel and 20 are industry personnel. 


An example of how individual responses are recorded is as follows . Under the 
questionnaire section covering [SCOPE OF PER COVERAGE! several practice areas 
such as SPECIFICATIONS, DOCUMENTATION, etc. are listed. Participants were 
asked to indicate their judgment of the cost impact of the practices used 
in each practice area on the total cost of space programs as H-high, M- 
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moderate or L-low. Three rows of numbers are used to record cost impact 
responses.' "The top row of numbers reflect NASA responses under H, M, L and 
T 1 for the total number of NASA answers received. The second row reflects 
industry response under the same cost impact headings. The third row shows 
the totals of the first two rows when added as columns under H, M, L and T 1 . 
Thus, the cost impact responses recorded under SPECIFICATIONS are 

H M L T 1 

NASA 20 5 2 27 

Industry ' lb 5 1 20 

Total 3^- 10 3 bj 

Participants were also asked to indicate a priority of preference for PER's 
covering each practice area. The same arrangement of rows and columns is 
used to record NASA and industry responses separately and combined for each 
practice area. Priorities 'are ranked from 1 to 5 with 1 indicating the 
highest priority. Columns ’5 includes priorities 5 and above as very few 
indicated beyond the fifth level. 

Participants were also asked to indicate which of several sub-topics they 
believed to be important within each practice area. In these cases, only 
one row of numbers is used to indicate all responses. The first and second 
numbers indicate NASA and industry responses respectively and the third 
reflects the total of the two. This number arrangement is used in all 
remaining cases where a single row of numbers could be used to adequately 
reflect the quantities of responses received. Elite type has been used to 
identified "write-in" type responses. 
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QUESTIONNAIRE 

PROGRAM EFFECT RELATIONSHIPS (PER'S) 

Question Topics Covered 

1. Scope of PER Coverage 

. | 

2. Content/Format of PER's and Handbook 

3. Usage of PER's and Handbook 

j 

4* Method of Ge ne rating A/a! idating PER Handbook 
5; Low-Cost Practice I mplementation Considerations 

6. Availability of Data for Current/Future PER's 

7. Opinions on Current/Future Program Practices 



LOCKHEED MISSILES 



& SPACE COMPANY, INC. 
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Special Notes : • 

(1) Answers to this questionnaire are being acquired and assembled for the 
NASA/HQ, Low Cost Systems Office, by Lockheed Missiles & Space Company 
as part of a current study contract on Low-Cost Program Practices . 

(2) This questionnaire package is being sent to allow time for preparation 

prior to the planned, personal interviews. For orientation and back- 
ground relating to the questionnaire, two supplementing documents are 
being supplied : " 


(a) LMSC-D387543 dated 4 October 1974, "Program Effect 
Relationships (PER's) - Survey Orientation 

(b) LMSC-D387518 dated 30 May 1974, "PER Handbook - 
Appendix 1, Final Report on Low-Cost Program 
Practices" 

(3) It is the intent of the survey that all answers be as candid and objective 
as is feasible. Therefore, if it is desired by the interviewee that his 
responses be treated in an anonymous manner in any published summary 

'report on the survey, this will be done. 

(4) This PER survey and analysis of results will provide NASA/HQ with suffi- 
cient data upon which to base further action on PER development and 

NASA Center/industry participation. 
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PER SURVEY QUESTIONNAIRE 


| Person Answering 


Name : 


Date of Answer : 
Title : 


NASA Center 
or Company 


Orgn . Name 


Divis ion : 


No: 


Phone 


Mailing Address 


Anonymity Requested 


[LMSC Contacts : 


□ Yes 

8 + 8 = 16 


□ No 

15 + 9 = 2k 


( a ) please direct all inquiries or special information by telephone to LMSC 

Dale Hannaford Orgn. 61-50 ( I *0 8 ) 7^2-8207 

Keith Burbridge ' < 4o8 > ^-1439 

( 408 ) 743 - 18^5 

Wayne Miller v 

(b) For information that must be mailed, please address to: 

Wayne Miller, Advanced Payload Systems 
Lockheed Missiles & Space Company, Inc. 

Space Systems Division 
Orgn. 6l- 50, Bldg. 104 
P, 0. Box 504 

Sunnyvale, California 94088 
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SCOP E OF PER COVERAGE 
1#1 principal P™r„m Practice areas an a .uMopl c s^ houia^g^B^. 

in new PER development ? 

(1) Mark high "H", moderate "M”, or lo» ”L" for estimate of cost 
impact of practice on spacecraft programs, 

• ( 2) - Mark lumbers 1, 2, 3, etc., for priority of preference for FER's 
covering the practice areas , 

(3) Mark an "X" in the subtopic block(s) which is/are belxeved 
most important . 


Cost Impact 


H. 

M 

L 

T 

+ 

o 

CVJ 

5 + 

2 = 

27 

l4 + 

5 + 

1 = 

20 

34 

10 

3 

47- 

6 

20 ■ 

■ 3 

29 

5 

13 

2 

20 

11 

33 . 

5 

49 

5 

11 

11 

27 

11 

6 

3 

20 

16 

17 

l4 

47 


■ Practice Area 


Priority 

I II III IV V 


11 

13 


4 

4 


24 8 6 


5 3 

7 6 


SPECIFICATIONS „ 

— j 9+8=17 

dQty. Growth 3+9=12 □ Project Peculiar 

□ Redundancy, Interrelations 10+16=26 

□ Level -Applied (Commercial, MIL, etc . )l9+9=28 

f-~| WORK IMPOSED 3+2=6 - 

DOCUMENTATION 

□ Qty . , Redundancy, Revisions 22+16=38 
i I Type , Degree 1 ^ ^ ^ □ Format 2+o=8 

□ control and Classification 7+7=14 

□preparation vs Reading 7+4=11 
f-| SCOPE OF COVERAGE 1+3=4 

CONTRACTING PRACTICES 

□ Procurement Procedures 13+12=25 xx+10=21 7 s 4 

ASPR Influence 2+5=7 ^ ? 

S 7+5=12 

Extensive Boiler Plate 
_ 8+7=15 

□ impact on Contractor Overhead 


3 


7 


|~7 QUANTITY OF INTERFACES 0+3 =3 


5 - 
1 


3 

4 


:F|/g° P ° sal J l ir -§ 

□ Contract Negotiation 


1 4 
0 0 
1 4 


4 9 

0 0 


12 9 7 4 9 


2 9 

o o 


2 9 
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Cost Impact 
H M L - -T 

26 0 3 29 

19 4 1~ 20 

4l 4 4 49 


7 13 6 

4 9 4 


26 

17 


11 22 10 43 

12 13 3 28 

- l4 2 1 17 

26 15 ' 4 ' 45 


10 13 5 28 

9 8 3 20 

19 21 8 48 


Priority 


Practice Area 


PROGRAM DEFINITION 


26+15=41 


8+6=l4 


I 

13 

13 


II 

fr- 


ill 

4 


IV V 
1 2 
1 0 


I | 

LI New Technology Development 26 9 6 2 2 

"] *1 r 1 ■ 

□ Phased Project Planning □ Scheduling 8+8=16 

□ Contingency Allowances 9+9=18 □ Inflation Allowance. 6+7=13 

WHEN FINALIZED? 1+1=2 ; 


11+2=13 


□ Mgmt. Planning 

10 4 


SYS. ENGR. & INTEGRATION (SE&I) 

□ value of Functions 12+6=18 
p| COST/WORTH EVALUATION 3+1=4 

NASA PROGRAM MANAGEMENT 
□ Mgmt . Risk 16+5=21 

□ Mgmt ."^rocedures/Cantrols (C-spec, WBS) 

□ contractor Interfaces 9+10=19 

□ N^SA^Contractor Personnel Functions/Ratios 

□ Review/Approval Cycles 16+8=24 

□ ____ 

R/QA 

□ Level of Reliability Analysis/Test 17+11=28 

□ Degree of QA, Inspection 10+10=20 

I~1 NASA/ Contractor QA Redundancy 7+12=19 

□ Hardware Risk Acceptance 15+8=23 
f~l Redundant Supplier Audits 4+6=10 

□ Value of R/QA Functions 11+6=17 
p-[ REVIEW BOARD CYCLE TIME 1+1=2 


3 

3 


5 

8 


5 

4 


19 • 8 


3 

4 


13 7 


4 

6 


4 

2 


0 11 
0 1 


10 0 12 

2 2 7 

10 0 


2 7 


3 7 

2 0 


5 7 
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i 


jl 


Cost Impact 


H M L T 
4 17 7 28 

9 11 4 20 


9 28 11 48 


10 16 3 29 
12 6 2 20 


22 22 5 49 


12 13 3 28 

2 8 4 14 


14 21-7 42 


7 11 

6 6 


6 24 
3 15 


13 17 9 39 


2 12 12 26 

l 8 ll 20 


3 20 23 46 


Practice Area 


GROUND SUPPORT E SHIPMENT (GSE) 

— 19+10=29 

□ Level of Requirements (Commercial, MIL, Flight) 


□ Qtv. vs Location vs Scheduled Usage 12+9 21 
13+7=7 ~ 


l — - xj=r=20 

1 1 Commonality 


□ 


TESTING 


I I Component vs Subsystem vs System 20+9=29 
I iRedundant Testing 10+8=18 
□Te st Cost vs Program Success 15+8=23 
I I NASA vs Contractor Testing 4+4=8 
□ OVER THE SHOULDER SURVEILLANCE 1+2= 3 


ENGINEERING 

15+642T 


□ Value of Functions (degree of analysis) 
j | Reporting Formality & Quantity 1+2=3 


MANUFACTURING 


: 6+4=10 
Ll Processing 
n 11+8=19 
LJMfg. Planning 


4+4=8 

□ Tooling 


[ 1 Subcontracting 


3+7=10 

i I Mfg. Test 


□ Material Procurement 8+5-13 

□ 


MAINTENANCE AND LOGISTICS 

CU Type /Degree M 9+3=12 

□ Spares Qty . /Program vs Multi-Program Usage 

□ Field Repair/Refurbishment vs Discard 5+3=8 


14+11=25 


□ . 


(Other) 


(Other) 
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1 

2 

3 


Priority 
II III IV V 
2 6 2 11 
■12 310 


□ inventory and Reuse 10+7-17 
□ STE, AGE, FSE 3+3=6 


5 ■ 14 


4 

8 


7 

3 


10 


4 

6 


12 10 


3 

4 


2 

3 


2 

2 


7 4 


1 

O 


3 

7 


3 10 


6 

3 


l 

3 


2 

2 


6 

4 


10 


3 ll 


4 6: 

1 0 


5 6 


4 7 


l 3 


4 5 10 


0 1 
0 2 


0 3 


1 12 - 


2 1 


3 13 1 


i .CPP-l 2 
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! 


'} 

i 

i 

I 

fed! 


1.2 


Cost Information (vs program practice) on PER should be at -what level(s)? 


■ j .16+10=26 

a) 1 I Program 

□ 4+3=7 

Component 


I — j 11+10=21 

b) 1 1 Spacecraft 

a) □ FUNCTIONAL TASKS 


□ 21+10=31 
Subsystem 

1 + 5=6 


1.3 Cost information on PER should be in terms of what echelon of cost 
■ accumulation? 

a, □ Contractor costs only. 4+3=7 

b) 

c) EH NASA plus Contractor combined. 7+6=13 


□ NASA, Contractor separately. 17+11-28 




1.4 Hie practical range of cost vs practice on a PER should cover what 
classes of hardware? 

a) □ Commercial 9+^~-13 b) □ Commercial Aircraft 13+11-3^+ 

1 4+3=17 j — I 

c) I 1 Military Aircraft d) I — I Spacecraft 26+20=46 


e ) | | ALL 2+3=5 

1.5 Should PER's be limited to cover unmanned spacecraft only or extended 
to cover other hardware? 

a) EH Experiments or Mission Equipment 18+12=30 

■ - j 18+8=26 I — | 

b) I I Manned Spacecraft c) 1 | Space Shuttle 16+8=24 

d ) Q LIMIT TO UNMANNED SPACE PROGRAMS 1+2=3 


l6 
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CONTENT/FORMAT OF PEN'S and handbook 


LCPP-12 


2.1 Should PER data be limited to historical data accumulation or show 
estimated data for lower cost practices? 

a) I I Historical data only (like CER's). 2+4=6 

b) [[[j Historical data plus comparative estimates of using 

! 

lower |cost approach. 19+18=37 
=> □ Estimates for lower cost approach only. 1+1=2 
d ) [™] ( b ) PLUS CHANGING ECONOMIC FACTORS 1+2=3 


2 -2 Which data sources should be included in PER or User Instructions? 
a) 1 I Historical data. 14+10=24 

»> □ Rationale and/or derivation of low-cost practice estimate . ' 22+1, =39 

c) n 


2.3 


Types of historical data to be included in data sources or backup data? 

[——j 21+13=34 1— j _ - 

a) j 1 Specific programs b) | | Specific document (s) 8+3=9 


12 + 8=20 

:) j J Specific contract 


!)□ 


ALL 1+0=1 


2.4 What is preference for individual PER format? 

10+8=18 


a) 11 Line Graph 20+15=35 b) [[[] Table 

I — I 3 + 3=6 I — j 

d) I (Mathematical Equation e) | | Mathematical Equation 

f) r] ALL 3+4=7 


) □ Bar Chart 9+5-14 


2 . 5 User Instructions for PER ' s ? 


a) 1 1 Separate sheet with each PER. 10+8=18 

b) □ On face of PER. 9+7=16 

c) 1 1 All combined in one section of Handbook. 5+6=11 

a) PI AS REQUIRED FOR BEST USE 1+1=2 + 4 
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2 .6 For each of the ma,1or PER practice areas listed in Question No. 1.1, 
what specific, parameters displayed on PER would appear to be of 
■' greatest value in your -planning and estimating? For example, under 
"Specifications" : 

o Degree of Safety Certification vs Relative Cost 
o Maintenance Index vs Relative Cost 


a) Specification Parameters 

Med for "each spec . government & industry cost for prepara- 

( 1 ) PARAMETER. vs TION OF & FOR MEETIN G EACH PA RAMETER 

OVERALL COST OF NASA CUR- 

(2) RENT SPEC APPROACH. vs ALTERNATIVE LOW COST APPROACH ES. 


b) Documentation Parameters 

(1) QUANTITY OF vs MANAGE MENT EFFICIENCY 

COST OF EACH CHECK BY FUNCTIONAL 

(2) MULTIPLE DRAWING CHECKS vs ORGANIZATION VS. WORTH EVALUA TIONS . 


c) Contracting Practice Parameters 

(1) TYPE OF CONTRACT vs 

( 2 ) TYPE OF CONTRACT vs 

. . LEVEL '& "Frequency of Govern- 

(3) ME NT AUDIT /REVIEWS 

d) Program Definition Parameters 

DEGKEE^OMTjETE FOR EACH 

( 1 ) PHASE. vs 

COST OF COMPLETELY DEFIN- 

(2) IN G ALL TECH. & MGMT, vs 

“ASPECTS OF PROGRAM DURING 

r( t > 

e ) SE&I Parameters 

OVERLAP & REDUNDENCY WITH 

(1) OTHER GROUPS vs 

(2) COST OF SE&I vs 

f) NASA Program Management Parameters 

MAJOR POLICIES AND PRO- REAL & IMAGINARY COST IMPACTS ON 

( 1) CEDURES. vs PROGRAMS 

COST OF EXISTING TECHNIQUES AND 

(2) TECHNIQUES vs ALTERNATIVE TECHNIQUES 
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GOVERNMENT & INDUSTRY ADMIN. COSTS 
OVER/UNDER RUNS BASED ON 0 C/D 
NEGOTIATED COSTS . 

COST UF'TJAME VS PROGRAM OVEE/ONDER 
RUNS. 


NO. & COST OF CHANGES IN 0 C/D 

COST OF SURPRISES , CHANGES, REDESIGN & 

REMANUFACTURE WHEN PD IS FINALIZED IN 

JIT. ? ' P'V".;, 


COSTS OF OVERLAP 

TOTAL PROGRAM COSTS WHEN FORMAL SE&I 
IS USED & IS NOT USED, 


IM3C-D3875^ 

LCPF-12 



g) R/QA Parameters 

COST OF EXISTING PRAC- 

(1) TICES . vs PROGRAM GOST & SUCCESS. 

("RANGE OF R&QA COSTS FOR f DEGREE OF SUCCESS ACHIEVED IN 

(2D HISTORICAL PROGRAMS vsf EACH PROGRAM. 

h) GSE Parameters 

DEGREE OF DEFINITION & COST COMPEETED PROGRAM DEFINITION & 

(1) EST. AT END OF EACH PHASE, vs COSTS . 

PROGRAM COSTS WITH & WITHOUT 

(2) STD COMPONENTS (MAXIMUM) vs STANDARDIZATION. 


i) Testing Parameters 

(1) LEVEL & AMT. OF TESTING. vs PROGRAM SUCCESS & COSTS. 

(’REDUNDANT COSTS IMPOSED (MANHOURS, 

(2) REDUNDANT TESTING. vs^ EQUIP . . ' FAC ILITIES . TRANSPORT . ) 

0 ) ' Engineering Parameters 

CNO. OF ENGINEERING DISCI- 

(llf PLINES & ORGANIZATIONS. vs COST OF EACH & INTERFACE COSTS. 

(COST OF MEETING PRESENT" ('COST OF MEETING MINIMUM REQUIREMENTS 
( 2 y REQUIREMENTS . vs) TO COMPTETE MTSSTON. 

k.) Manufacturing Parameters 

COST OF MFG. PLANNING COST OF ACTUAL FAB, ASSY, & TEST 

(1) MANAGING & CONTROL vs EFFORTS . 

EXTRA COST DUE TO GOVERN- COSTS OF USING CONTRACTOR HOW-TO 

(2) MENT HOW TO SPECS. vs PROCEDURES WITHOUT GOVERNMENT S PECS. 

1) Maintenance and Logistics Parameters 

SPECIAL REQUIREMENTS BY EACH 

(1) CUSTOMER . vs EXIST I NG OOMIRACTOR PRACTICES. . 

COSTS DUE TO PRESENT SPEC 

(2) REQUIREMENTS . : vs ALTERNATIVE LOW-COST PRACTICES. 

m) (Other) CONTRACT MONITORING P arameter 

COST OF USING OF PREPLANNED" , vs MONITORING BY THE "BIG TEAM" 

(I) MONITORING QUESTIONNAIRE S & MEETINGS & EXCESSIVE FORMAL 
QTRLI . PERSONNEL CONTACT'S , WKLY . DOCUMENTATION NOW BEING USED . 

PHONE CALLS BETWEEN A FEW 
RESPONSIBLE INDIVIDUALS. 

n) (Other) MEETINGS RECORD FORMS Parameter 

RECORD ALL NASA & INDUSTRY COSTS ASSOCIATED WITH EACH MEETING 
(1) & REVIEW & SHOW WHAT DECIS DDNS ARE MADE, twpt; FMENTFR , CFfANGFR OR 

POSTPONED AND GIVE REASONS $HY. 


19 


LOCKHEED MISSILES & SPACE COMPANY. INC. 


1 * 480 - 03875 ^ 

LCPP-12 


2 .7 In -what form should the relative cost of alternate practices be 
provided on a PER? 

$ of Total Program Cost 11 + 6=17 

Cost factors (multipliers) 9 + 6=15 

Delta cost among alternatives 17+12=29 


□ 

□ 

□ 

□ 
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3. USAGE OP PER'S AND HANDBOOK 

3*1 Xs there a, preferred organizational level or personnel to which 
primary usage of PER's be directed? 

a) □ Planners 8+3=11 b) □ Program Managers 23+14=37 

=> □ Estimators 10+7=17 d) □ Senior Management 13+4=17 

e) □ Functional Organization: 10+12=22 

□ Contracts 5+7=12 □ Finance 2+7=9 

□ Manufacturing 5+7=12 Dr/qa 7+7=i4 

□ Procurement 9+6=15 □ Engineering 10+10=20 

□ Testing 5+3=8 □ Operations 3+5=8 

□ _ □ 

*■) □ (Other) 

3 .2 What is primary use(s)_ contemplated for PER's? 
a) □ Supplementing other estimates. 7+6=13 

k) 1 — I Evaluating estimates made with CER's, bottom-up or other means. 5+4=5 
• c ) I — I Basis for primary program decisions . 17+6=23 * 

d) □ For general cost tradeoffs . 18+9=27 

e) j I AVOIDING UNNECESS ARY COSTS 1+3=4 

3*3 Could PER s developed primarily for NASA usage be also of value 


for use by Industry? 

j J 8+6=14 I — - 15+11=26 

a) 1 j Most b) j j Some 


.)□££*■ a) □ 
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What organizations in your facility would use PER's if they were 
available? 


rr 1 7 + 4=11 
a; 1 — I Top Management 


I — I 18+11=29 

b ) l (Program Managers 


c) EZ] Functional Organizations (list) 14 + 9=23 

ENGINEERING 3+5=8 MANUFACTURING 0+3=3 

R&QA 2+2=4 ESTIMATING 0 1 - 3=3 


TEST & DPS 1+2=3 


a) d 

e)Q 

List potential limitations or reservations for usage of PER's or 
Handbook at your facility . 

□ _ LACK OF ACCURATE/ADEQUATE HISTORICAL DATA J 

DATA CREDIBILITY- 

□ PEOPLE DON'T READ NOR USE EXISTING HANDBOOKS 

J R ELUCTANCE TO CHANGE SAFE & COMFORTABLE PRACTICES EVEw’tf tttov 

ARE HIGH COST 
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METHOD OP GENERATING/VALIDATING PER HANDBOOK 


1 


Is a NASA-wide PER Handbook considered feasible ana desirable? 

r— i 15+12=27 I I 10+3=13 
a) I 1 Yes b) LJ No 


2 What basic method(s) recommended for NASA preparation, issuance, 
maintenahce of PER's? 

NASA/HQ prepare a general usage set of PER's. 11+7=18 

» □ Each NASA Center prepare selected groups of PER's, 
coordinated by NASA/HQ. 9+4=13 

=) □ Each NASA Center prepare a separate PER Handbook for its 6+3=9 
own use. (Send copies to other Centers for information.) 

DON'T DO IT 1+2=3 



3 To what degree should Industry contribute to or participate in 
PER preparation? 

□ 22+11=33 v n 6+7=13 , r— i . rn 

Active b)l — I Advisory Only c)| I None d) j |_ 


k Will PER's showing only relative cost factors (or jo of total program 
cost) for alternate program practices be a significant planning tool 
even though they cannot be used to sum the total cost of a program 
element in absolute dollars or man-hours? 

| — ( 22+13=35 . I — I 6+3=9 

a) I I Yes b) | | No 

5 Is actual validation on a sample program!' s) required before general 
usage of a PER or sets of PER’s is implemented? 

a) DyS* 15 *) □ No 17+13=3 ° 
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What type /level of validation is required before new low-cost practice 
PER's should he applied to new programs? 

a) □ Actual proof on 1- to ^ (qty.) pilot program(s) with 

low-cost practices applied . 11+4=15 

b) □ Verify logic and probability of significant saving with 

low-cost practices by theoretical calculations (cost tradeoffs) 

on specific program before application of PER's. 9+6=15 

o □ No prior validation; apply lowest cost practice which 

is judged to meet the mandatory-minimum program require- 
ments (Program Manager decision). 8+9=17 

d ) □ (Other) WHY BE CONCERNED WITH VALIDATION NOW? EXISTING 
PRACTICES GREW WITHOUT FORMAL VALIDATION. 
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5. 

5.1 


LOW-COST PRACTICE IMPLEMENTATION CONSIDERATIONS 


Do you feel current lavs and NASA regulations (ASPR, etc.) must be 
changed to allow implementation of low-cost practices? 

a) 

□ Which ones? ■ ASPR 


□ 7+11=18 I — I 13+4=17 

Yes b ) 1 1 No 


CHANGE CONTRACTING APPROACHES AND REGULATIONS 
1 — I What primary changes? FROM THE PRE-WWII ’’HOUSEKEEPING" SUPPORT P HTLO- 

• SOPHY REQUIRING SPACE PROGRAM CONTRACTS TO BE EXCEPTIONS TO INCLUDE 1970 - 
19g0 STYLE ^STREAMLINE REQ'S. FOR ACQUIRING LOW-COST NO EBILL S &RAC E 

PROGRAMS . 

5 • 2 Do you believe PER's can be an effective method for aiding near-term ' PROGRAMS . 

transition to low-cost practices and lover-cost space programs? 

I — I 18+13=31 v I — j 6+4=10 

a) 1 j Yes b) | | No 

5* 3 Can you suggest a better basic method or "tool" for achieving lower 
space program costs? 

• □ CHANGE TOP LEVEL POLICIES & PROCEDURES TO DEMONSTRATE LOW-COST 
LEADERSHIP . 


□ . 


5 • 4 What principal areas of program. practices appear to have the highest 

probability of actual change to low-cost approaches, and in what order? 


□ STANDARDIZED SPACE HARDWARE 


□ .REDUCTIONS IN TESTING. R&QA. SPECS fc DOCUMENT A r T l Tm\T 


Q IMPROVED PROGRAM DEFINITION 


[~j DEDICATION OF HIGH LEVEL LEADERSHIP TO LOW COST PRACTICES IN FUTURE 
PROGRAMS. 
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5 . Wh§t_ organizational level of authority would be desirable or necessary 

for di recting implementation of low-cost practices ? 

Cl NASA: TOP HQ MGMT & FIELD CENTER DIRECTORS 

□ Industry: TOP COMPANY MGMT. 


5*6 For max imum impact. on program cost reduction, should PEH's or PER 
, Handbook be applied as ; 


NASA Contractor 


D Set of guidelines for program 

Q 21+13=34 

□ 18+9=27 

Cl Firm operating requirements document 

Q 4+3=7 

□ 2+2=4 

Cl Formal contract requirement 

□ 0+3=3 

□ 1+4=5 

□ 

□ 

□ 


5 Zog_°&xiMUM impa ct on program cost reduction, where within NASA and 
Industry should prin cipal responsibility for implementing low-cost 
practices be assigned? 

a) NASA 

f— I Autonomous NASA Centers 6+7=13 
, ^ ^ Centralized NASA Function (Lead Center) 2+1=3 

•: D nasa/hq 10+6=16 

n 

I — I Individual Program Offices 15+3=18 

□ 

b ) Industry 

Corporate Management 6+4=10 
Company Management 6+2=8 
Division Management 7+6=13. 

Individual Program Offices 12+9=21 
.Individual Functional Organizations 6+2=8 
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□ 
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6. AVAILABILITY- OF DATA FOR CURRENT / FUTURE PER'S • 

6.1 What data pertinent to program practice area or sub-practice vs cost 
impact are now available for potential inclusion in PER nets or in 
Handbook? 


Program Practice (vs Cost Impact} 


m 533 FORMS' & POP’S 


2| PMR’S 


Document or Source 


LU CONTRACT FILE S • 

. . PERFORMANCE SPECIFIED VS . DEVELOPMENT 

Li] COSTS . 

HISTORICAL DATA NORMALLY NOT RECORDED TO 

|T| LEVEL OF DETAIL REQUIRED . 

6.2 In what status is "available" data? 


□ CONTRACT FILES 

NOT IN NASA FILES 
|~1 NOT IN CONTR FILE S 


□ Dead storage (not readily retrievable). 2+3=5 

□ Readily retrievable from central files , 0+2=2 

1—1 Readily retrievable from program files. 6+1=7 

□ Hard copy available . • 4+0=4 

| | LIKE PULLING HEN 1 S TEETH 1+4=5 


3u feel that the NASA historical programmatic and cost records in 


your Center or Company are adequate for generating meaningful PER's? 
j — | 5+4=9 , 20+8=28 . — . 1+0=1 

I I Yes LJno 1 | ADEQUATE FOR JUDGEMENT LEVEL ANALYSES.. BUT 

NOT FOR QUANTITATIVE LEVEL. 

6.4 Could data (6.1) be made available for general use by: 


13+5=18 


Other Contractors 
7+0=7 


rnun a ki\/ 
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7 * OPINIONS ON CURRENT/ FUTURE PROGRAM PRACTICES ] 

7 ‘ 1 — * percentage of program cost reduction could be obtained by imple- 

menting low-cost practices ? 



7 ' ^ ~ y ha ' t principal areas are NASA Centers and Industry actively trying 

to reduce costs? 

I- — i Eliminating desirable but non -mandatory tasks. 18+9=27" 

a Reducing requirements . 20+4=24 

□ Technical 16+2=18 □ Mgmt . 13+2=15 □ Reporting 15+2=0 

□ _ Testing 3.+l=4 . □ 

L I Personnel utilization improvement 5+6=11 

n ■ ‘ 

— 1 Overhead functions/facilities 12+5=17 

HU Assuming larger risks 15+5=17 
HU STANDARDIZING HARDWARE 1+2=3 
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6.1.2. SUMMARY OF KEY RESPONSES 


Listed' Hereunder are' those responses 'to the Survey, which are believed to be 
representative prevailing opinions held by the interviewees. Of necessity, 
the answers listed have been generalized to allow simple categorization. Of 
particular interest is the predominating opinion that should Low Cost 
Practices be implemented, program cost reductions of the order of 25/ TPC 
would be possible. This view was held by over k0% of the respondees. 

For quantification purposes, the word "Other" is used to denote the percent of 
answers left blank and/or qualified to the' extent that they were not deemed 
applicable responses to the question asked. 


1. What percentage of program cost reduction could be obtained by implement - 
ing Low Cost Practices? 


% Savings 


0 ' 

10 

15-20 
25 
5° 

Other 

The remainder of the responses to the 15 questions posed by the Survey have 
been summarized in a similar manner, and follow on Pages 31 thru 35. 


% of h9 Responses 
2 

16 

10 

U3 

18 

11 
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|Th S following questions, and their accompanying responses, should not be 

considered as in order of importance. The order is only that in which the 

questions were asked. No attempt has been made to weight the responses 
statistically ; 

2 ' — y0U fe - 1 that the historical programmatic and cost records in 

y our Center or Company are a dequate for generating meaningful PER’s ? 

Yes 18 # 

No 57 # 

Other 25# 

,J> ‘ S hould PER data be limited to historical data accumulation or show 
estimated data for lower cost practices ? 

Historical data only (like CERs) 


12 # 


Historical data plus- comparative 
estimates of using lower cost 
approach 

Estimates for lower cost approach 
only 

Other 


' What is primary use(s) contemplated for PER's? 




* 


Supplementing other estimates 27# 

Evaluating estimates made with CER’s, 
bottom-up or other means 

Basis for primary program decisions 47# 

For general cost tradeoffs 33% 

Other od 


for S pER^s. 10-1 lndlcates some participants contemplated multiple uses 
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cj . Could PER’ s 

developed primarily for WASA usage be also of value for 

use by Industry? 


Yes 

82 $ 

Wo 

4$ 

Other 

l4$ 


6 . Do you believe PER's can be an effective method for aiding near-term 
transition to low-cost practices and lower-cost space programs? 


Yes 

63$ ' 

Wo 

2$ 

Other 

35$ 

7 . Will PER’s 

showing onlv relative cost factors (or $ of total program 

cost') for alternate program practices be a significant planning tool even 

though thev cannot be used to sum the total cost of a program element in 

absolute dollars 

or manhours? - 

Yes 

71$ 

Wo 

18 $ 

Other 

11$ 

R -Is a NASA-wide PER Handbook considered feasible and desirable? 

Yes 

55$ 

Wo 

27$ 

Other 

18 $ 


What basic method (s.) recommended, for NASA preparation, issuance, 
maintenance of PER ' s ? 
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NASA /HQ prepare a general usage set of PER's 37$ 

.Each NASA Center prepare selected g-roups of - 
PER's, coordinated by NASA/HQ, 27$ 

Each NASA Center prepare a separate PER Handbook 

for its own use. (Send copies to other 

Centers for information. ) 18$ 

Other 18$ 


’What is preference 

for individual 

PER format? 


Line graph 

72$ 

Mathematical equations 

CM 

H 

Table 

37$ 

All forms OK 

l4$ 

Bar chart 

29$ 




User Instructions for PER's ? 

Separate sheet with each PER 37$ 

On face of PER 33$ 

All combined in one section 

of Handbook . . > 22$ 

Other 8$ 


Which data sources should be included in PER or User Instructions ? 

Historical data 49$ 

Rationale and/or derivation of 

low-cost practice estimate 80$ 


Types of historical data to be included in data sources or backup data ? 


Specific programs 

69$ 

Specific document (s) 

18$: 

Specific contract 

12$ 

All applicable 

2$ 
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lU. To what degree should Industry contribute to or participate in PER 
preparation ?' - • 


Active 

67 % 

Advisory only 

27 % 

Other 

6 % 


15 • .Can you suggest a better basic method or "tool” for achieving lower 
space program costs ? 

Thirty-five percent suggested other general approaches , but these 
did not include specific "tools." Suggestions included: 

a. Choose Project Managers based on past (low-cost as well as 
success) performance 

b. Have top management dictate less-detailed management while 
continuing to mandate lower budgets 

c. Make achieving low costs a challenge to Project Managers, not 
a set of detailed rules from upper management 

d. Establish early,- complete and firm program definition 

e. Use only fixed-price contracts 

f. Create a vehicle to continuously spread the low-cost word 
...... • 

g. PERs are only a start — all implications must be understood, 
including program and personal (career) risks 

* h . Use PERs or something similar in a mandatory educational program 
for key program management and administrative personnel 

i. Use MIL-STD-499 in complementary role with PERs 

j. Concentrate on cost-effects of top level policy 

k. NASA managers need motivation, direction, and effective examples 
from top management relative to low-cost performance 
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!• Simply give a Program Manager a fixed dollar budget and make clear 
his job depends on not exceeding it 

m. Convince everyone involved '(NASA and Industry) that low costs are 
mandatory; then let managers and workers devise their own cost 
reduction approaches 

n. Establish absolute cost ceiling on each and every major cost 
element of a program 

o Hold each functional manager (technical and administrative) 
responsible for achieving low costs in his area. 


6.1.3 PER LIMITATIONS AND RESERVATIONS 

NASA and industry limitations and reservations concerning development and use 
of PERs were too extensive to be shown in the questionnaire space provided. 
They are summarized below. 

NASA : 

a. People do not read and will not use handbooks to manage programs 

b. Generalized PERs will not be applicable to specific programs 

c. Historical data does not tell whole story; therefore, PER con- 
clusions becomes one man’s opinion 

d. Just another control from above 

e. Will be much harder to establish confidence in PERs than in CERs 

f. People in a position to influence NASA-wide and field center 
policy will, not read nor use a handbook 

g. Lack of (quality/quantity of historical) data to generate 
meaningful PERs 

h. Even if low-cost practices and PERs were "perfect tools" they 
would not be used in must succeed programs nor where major risks 
are involved 

i. Must have mandatory training program and strong direction from 
HQ to all levels of NASA management if they are to be used 
effectively 
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Difficult to keep PEHs current to reflect low-cost progress 
being made 

Inertia of high level administrative and management people 
will prevent usage 

Low-cost approaches are considered to be uncomfortable and 
high-risk approaches 

NASA HQ want field center managers to take more risks, but do 
not provide quantified direction nor clarify acceptable "con- 
sequences relative to program success and/or individual careers 


PERs /Handbook will not be used by industry if their use conflicts 
m any way with a specific RFP and/or proposal evaluation 
instructions 

Historical data is meager , non-definitive and sometimes purposely 
deceptive and will result in questionable conclusions 

Low-cost approaches and PERs that work well for one company will 
not work at all for other companies without completely reorganizing 
personnel and revamping company policies and procedures 

Lack of top level support of low-cost practices results from 
success in previous Tiigh-cost programs and overhead practices 

PERs must identify risk vs. cost savings to all government and 
industrial organizations involved 

Different sets of PERs would have to be developed for many types 
of program situations 

o New programs with mostly new technology 
o New programs using only known technology 
0 New programs using standard hardware 
o Follow-on programs with high inheritance 
o Follow-on programs with low inheritance 

o Programs requiring long-term vs. sh-ort-term operational life 

A major limitation of PER usage will be establishment of confidence 
m ohe validity of costs associated with various program oractices. 
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12.6.1.^ Preferred Format For PERs and Handbook 

The general format preferred by survey participants is essentially the same 
as that developed in the previous Low Cost Program Practice Study. Line 
graphs, tables, bar charts and mathematical equations, in this respective 
order, are the formats preferred for PERs. Users instructions on separate 
(facing) sheets or on the face of the PER (when possible) are preferred over 
the concept of combining all instruction in one section of the handbook. The 
latter would require users to spend too much 'time' "flipping and hunting". 

Most participants preferred the use of a combination of historical data plus 
estimating data for generating PERs to the use of either historical data 
only or estimating data only. All preferred to have source data included on 
the- PER or in the users instructions. 

In retrospect the question-of using .historical data only for PER development 
could be considered somewhat i&practical. Historical data can be used only 
to quantify traditional high cost practices. Estimating data, estimates and 
judgments must be used to quantify potential cost impacts of alternative low 
cost practices . The true value resulting from this question is that 79$ of 
the participants recognized and accepted this situation as the preferred 
mode of operation. 

12 .6.2 Additional Comments and Observations 

The following is a compilation of comments orally expressed by participants 
during orientation presentations and interview sessions . All of these are 
related to achieving low-costs in future NASA programs. Some are extensions 
of PER survey questions and/or answers. Some express different ideas for 
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achieving low-costs. Both positive and negative throughts are included 1 

. ‘ ' ' | 
because in the opinion of most participants both positive and negative ; 

actions will be required to change present practices to low-cost practices j 

and depending on one’s point of view, a specific idea can be either positive . > 

or negative. j 

12.6.2.1 Comments About PERs 

: 

Three out of five NASA organizations surveyed basically considered a NASA- j 

wide .PER handbook to be feasible and desirable. One seriously questioned 
the desirability of such a handbook but did not question the feasibility. 

One considered it highly desirable but seriously questioned the feasibility. 

This Center's experience in working with historical program data (NASA and 

4 

contractor) has led to the belief that most NASA's past and present program 
practices cannot be accurately quantified in significant depth using the 
historical data available. All organizations questioned the depth, conti- 
nuity and credibility of historical cost and progrsmatic data to some extent. 

On the other hand, three centers felt that the PER approach is a good one 
and that PERs are very practical tools. These also believe that existing 
historical data is something less than perfect and that it will not be . 
possible to quantify all practices using existing data. Nevertheless, they 
recognize that existing historical data is the only "data" available for 
quantifying the cost impacts of program practices . In addition, they believe 
that the PER handbook approach will be of great value to NASA if as much as 
50 $ percent of the existing practices are • quantified with reasonable accuracy. 

They believe the shortcomings in NASA cost/technical record keeping system 
can be identified and corrected to allow accurate quantification of practices 

oq ftEPRODUCIBILIl.. 
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in future programs and continued PER expansion and refinement. In addition, 
they feel that the PERs generated to date are just what NASA needs to 
stimulate low-cost thinking and to offer challenging targets for future 
programs . 

The PER handbook is a desirable but not sufficient tool for implementing 
low-cost practices. Many NASA participants feel NASA should sponsor 
seminars and training courses on low-cost program planning and management 
in conjunction with or instead of a PER Handbook. Some feel that people 
will not use handbooks for planning and managing programs and point out 
handbooks and guidelines such as NASA's Statement of Work Handbook, NHB 
5800.1 and NASA's Phased Project Planning Guidelines are not now being used/ 
impler anted effectively. 

Industry in. general, believe a PER Handbook would be useful as a set of 
guidelines. However, industry* emphatically states that PERs will not be 
used if they conflict in anyway or degree with ideas expressed in RFPs , 
proposal instructions, proposal evaluation criteria and/or requirements and 
specifications contractually imposed as direct SOW statements or by 
reference to boiler plate specs. In short, industry will not risk having a 
major proposal eliminated from competition as being non-responsive to the 
above contracting instruments. In cases, which frequently occur, where these 
contractual instruments and/or their interpretation leave doubts as to the 
amount of work, extent of work overlap between functional work groups, 
degree of check and balance and scope of reporting desired by the NASA, con- 
tractors tend to be overresponsive as a hedge against a non-responsive ruling 
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The basic rationale being that NASA can negotiate oat excesses proposed but 
cannot legally "read in" implied nor logically intended responses which are 
not explicitly stated in proposals. 

12.6.2.2 Historical Data Comments 

o Historical programatic/cost data are often incomplete and misleading 
which in turn creates data credibility doubts and leads to the 
, feeling that PERs are _ just one person's opinion of data which can 
be interpreted several ways . 

o Most programs evaluated to date were completed prior to 1973 and do 
not accurately reflect cost impacts of NASA's here and now practices 
which to some degree are changing toward low-cost practices. 

12 . 6 . 2 . 3 Risk Considerations 

Many NASA participants expressed the feeling that spending less and doing 
less 'on programs would atvbomatically increase program risks. These also 
felt that PERs quantifying the cost impact of program practice alternatives 
wo'uld! not be used until the risk impacts of each alternative were also 
quantified. Others in NASA and industry feel that many program practices 
can be altered without impacting risks in any way. They also believe that 
some practices which do impact risks can be altered to reduce both cost and 
risks. In addition, they point out that the degree of risk due to existing 
practices have not been quantified to any credible degree and that presently 
there is no correlation between the amount of money spent in historical 
programs for assurance and/or risk reduction functions to the degree of 
program success achieved in space operations. 
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12.6.2.4 Program Definition 

Lack of early and complete program definition (including definition technical 
and management requirements, specifications to be imposed, program constraints 
and priorities) has been a major cost driver in most NASA programs. This 
belief was generally expressed by most NASA and industry participants. A 
few indicated that about the only thing which remains fixed after a phase 
c/d contractor has been selected is the program name and the costs initially 
estimated/ negotiated (even then the name may change). Most believe that 
NASA should concentrate on developing better and more complete progr am 
definitions before phase c/D Request for Proposals are written and issued. 

Some feel that KPP development should be an integral part of NASA's program 
definition efforts because last minute haste in RPP preparation tends to 
force program waste by eliminating time for cost/worth analysis of require- 
ments imposed, heavy reliance ©n boiler plate material and errors of omission. 
They feel that low cost practices will not be proposed Unless future EFPs 
clearly state that they are desired and give some indication as to the 

. a 

extent they will be tolerated in future proposals without risk of having 
such proposals ruled non-responsive . 

Some believe that NASA should use its program definition process, in addition 
to the Low Cost Systems Office (LCSO) efforts, to 

a. Identify high leverage low cost practices most applicable to the program 
being defined, 

b. Streamline/ eliminate how-to” specifications and procedures which are 

110 longer needed or are not cost effective with respect to the program 
being defined, 

4l 



lockhe; E d MISSILES & SPACE COMPANY, INC. 



LCPP-12 


c. Identify the minimum level of documentation, reporting and interfacing 
necessary for the program and the minimum cost method(s) of achieving 
the desired level, 

d. Evaluate and quantify the need for and cost-effectiveness of using 
duplicate management teams and duplicate technical support teams within 
NASA and contractor companies for the same program, 

e. Define and quantify the -most cost-effective ."single team concept" using 
a combination of NASA and contractor personnel to perform all necessar, 1 
functions with a minimum of overlap and/or duplication. 



LOCKHEED MISSILES & SPACE COMPANY. INC. 


LCPP-1 2 


12.6.2. 5 SPECIFICATIONS 

Common- belief expressed by many industry and some NASA participants relative 
to reducing the cost impact of specification are: 

• MSA should eliminate all specs telling contractors "how-to" do the 
job and issue only practical on-orbit performance specifications. 

• NASA should streamline and/or eliminate specs requiring multiple 
tiers of over- the- shoulder checkers to police/monitor work accom- 
plished by the initial doer and the first checker. 

• MSA should constrain spec writers by having them perform cost/ 
worth impact analysis before new specs are issued. 

• NASA should also constrain the proliferation of specs being accom- 
plished by rewording and/or combining general- off- the -she If spec's 

- into new program-peculiar specs without making significant changes 
nor voiding the previous general specs. These practices are expen- 
' sive. They can cost as much as $50,000 to $60,000 per spec, but add 
very little or nothing to the success of a program. NHB 5300.4 
(iP-l) is one such example. 

126.2.6 DOCUMENTATION 

Most participants feel historical documentation practices have not only been 
excessive but have also been ineffective. They feel that documentation 
requirements are not well planned early enough nor complete enough during 
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Phase B activities. This lack of requirements planning not only encourages the 
initially proposed list of documentation to be excessive due to contractor 
non-compliance concerns but historically it has allowed/ encourage the total 
amount of documentation generated during some programs to grow by an approxi- 
mate factor of two over what was initially proposed. On the other hand this 
practice does not provide the preplanned continuity and depth of information 
necessary to manage programs cost-effectively not to accurate evaluate cost and 
programatic impacts of technica’l and management decision and/or program 
practices from historical records. 

One NASA participant observed that the total amount of documentation purchased 
-in historical programs has been more directly proportional to the degree of 
government involvement (numbers of centers and people) than to anything 
resembling true program needs. Several others agreed with this observation 
and believe it is largely due to the practice of incrementally imposing/accept- 
ing documentation requirements/desirements NASA and industry people want to 
add for political as well as technical and management reasons. These, of 
course, feel that NASA should alter this practice and start imposing documenta- 
tion requirements according to a well defined documentation plan for each 
program based on the minimum, needs of the program. Recognizing that some 
degree of documentation is necessary for effective management and control of 
programs which do not get into development/operational troubles and that some 
additional documentation may be necessary for fact-finding and recovery purpose 
for programs which do experience such troubles these participants still believe 
NASA current documentation practices result in excessive documentation and 
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costs for either or both situations. Basic fixes recommended were to: 

a. Develop a minimum documentation requirements plan for general use 
in unmanned space programs (new and follow-on) via LCSO efforts. 

b. Develop minimum documentation plans for each specific program via 
MSA and contractor program definition efforts using the above 

•general plan for unmanned space programs as guidelines. 

c. Incorporate the minimum plan thus developed as a "not to exceed 
target" in the PFP's for Phase C/D portions ’of each program. 

12.6,2.7 TESTING 

The practice of pulling more test work from contractor plants to MSA centers 
as contract funding decreases is believed to be a minor cost driver by MSA and 
several contractors when existing test facilities and equipment can be used 
at ooth locations. The practice of performing redundant and or parallel 
testing at multiple locations which require additional test articles, equip- 
ment, personnel, etc., was considered to be a major cost driver in some 
programs . 

One significant low-cost alternative suggestion relative to testing is to 
standardize testing equipment and instrumentation (commercially available 
preferred) as much as possible. Others were to design to minimi ze/eliminate 
as much testing as possible and design so necessary testing can be performed \ 
by low- cost techniques. 
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12.6.2.8 PERSONNEL UTILIZATION 

Recognizing that at least 60 to 65 $ of all MSA expenditures are spent for 
labor and labor benefits, many participants believe MSA must develop more 
efficient personnel utilization practices at all levels of all MSA and 
Industry organizations involved in MSA programs. Although efficiency evalua- 
tion and quantif i vation of MSA/industry organizations/personnel is not an 
integral part of this study many participants believe it is the key to achiev- 
ing more space for each MSA dollar. Some sugges't'that MSA Headquarters 
should separate NASA research functions (personnel-associated costs) from 
program and program- support functions (personnel- related costs) and take the 
lead in defining more efficient roles for all MSA personnel. They also 
believe more efficient roles must be defined for MSA and contractor personnel 
for future programs in context with the single team concept previously men- 
tioned. The basic rationale supporting this suggestion is that talented 
personnel freed from desirable but semi -productive or non-essential tasks 
in traditional programs can be used, along with dollars saved by implementing 
low- ccvst practices to implement additional new programs within the limits 
of MSA "fixed budget." These participants feel that approximately 4o to 50 c f> 
of NASA and Industry personnel are already motivated to buy more space per 
NASA dollar in this manner, but can't implement their ambitions within the 
existing system of controls, constraints, habits and practices. One observa- 
tion commonly expressed is that in small programs, motivated people are 
successfully implementing short cuts and some low-cost practices without 
increasing risks. This brings up the question of why can't the same or even 
greater variances from traditional practices be implemented in large programs 
with comparable success? 
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12 . 6 . 3 CONCLUSIONS 

6.3.1 HISTORICAL DATA FOR PER' s 

a. Based on the historical data IMSC has analysed to date and inputs from 
survey participants, NASA historical programatic/cost data is not 
adequate to quantify a full set of PER's covering all major cost 

' deriving practices . • 

b. Historical programatic/cost data is frequently incomplete, inconsistent 
and misleading due to previous record-keeping practices used in all 
phases of programs, 

c. Historical data from programs completed prior to 1972-1973 if used as 
a base for PER's, would not include cost reductions being incorporated 
in more current programs due to tight budget constraints. 

d. Only 18$ of the survey participants believe the historical data avail- 
able in their center/company is adequate for generating meaningful 
(accurately quantified) PERs and IMSC experience to date supports 
this opinion as a basic conclusion. 
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12.6.3*2 PERs and PER Handbook 

A. The PER/Handbook concept is basically a good one but the present state 
of NASA’s historical data and the general lack of confidence (NASA and 
Industry) in existing historical data tend to preclude quantification 
and acceptance of PER's covering most of NASA's high-cost driving 
practices at this point in time. 

B. LMSC concurs with the general belief expressed by survey participants 
that if a perfect and complete PER Handbook could be generated at this 
time is use alone would not be adequate for implementing low-cost 
program practices even though it could be a very useful tool in con- 
junction with training programs and seminars. 

C. To effectively cover the scope of future of NASA programatic situations 
currently envisioned, different sets of PER's would have to be developed 
which would isolate and quantify individual practices and the inter- 
relationships between practices for: 

o New programs with mostly new practices 
o New programs using only known technology 
o New programs using standard hardware 
o Follow-on programs with high inheretance 
o Follow-on programs with low inheretance 

o Programs with long-term versus short term operational life 
o Programs with refurbishable hardware vs throw away. 

D. Both cost and data constraints preclude generation of PER's for the 
above programatic situations at this time. 
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E. Generation of PER Vs as they are presently conceived and defined should 
be discontinued .for the present in favor of alternative approaches 
which could have more wide spread acceptance and be more cost-effective 
for NASA near term transition toward low-cost practices. 

F. Many historically high-cost driving trends associated with various 
program practice areas can be adequately identified and alternative 
corrective actions can be successfully identified and implemented with- 
out using PER levels of quantification. 

12.6.3.3 Preferred PER/Handbook Format 
The preferred format(s) for PER's and a PER Handbook is basically the same 
as that generated in- the previous LCPP study (LMSC D387518)* This informa- 
tion could be of value if and when PER development is resumed. 

12 ,-6.3.4 RISK Considerations - 

A. Risk considerations are^ now and will continue to be a major concern with 
respect to NASA's implementation of low cost practices. 

B. * Survey participants from industry tend to disagree with the somewhat 

general NASA belief that risks will automatically increase if any low 
cost practices are implemented. 

C. The requirement frequently stated/ implied by NASA personnel that risks 
must be accurately quantified for each potential low-cost practice 
before PER's would be used or low cost practices would be implemented 
is judged to be a natural delaying type response to changes in the 
existing status quo of program operations. Students and layman some- 
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what versed in the Transactional Analysis theory of* psychology recognize 
this requirement as a response from one's child-ego-state instead 
of his adult -ego- state . In effect it says, "I will do your difficult 
and seemingly impossible task if you will do an even more difficult/ 
impossible task first". 

D. The idea of quantifying risk impacts as well as cost , .ipacts of 
potential low cost practices is not considered practical. Historical 
risk impacts have not been quantified nor recorded as well as cost 
impacts have. 

E. Risk impacts of alternative practices should be evaluated as soon as 
possible at the judgmental level and where practical at the quantifica- 
tion level. 

F. Unuil demonstrated otherwise, industry will be concerned with the risk 
of MSA's spending less money for less space programs instead of spend- 
ing the same level of dollars for more space programs after low cost 
practices are successfully implemented. 

„ • - 

12 . 6 . 4 Recommendations 

*A. Discontinue development of PER's at this time. 

"B. Start immediate development of a Work Breakdown Structure which will 
allow program practices to be accurately quantified in future 
space programs . 

*C. Start efforts to evaluate risk impacts of potential low cost practices 
identified to date at the judgmental level supported by estimated 
and/or historically sustaniated examples were possible. 
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Use potential low cost practices identified in this and other studies 
in future RFP's.to get additional cost and risk data estimates during 
phase B and phase C/D proposals. 

Continue quantification of historical program practices using in-house 
NASA personnel (or LMSC personnel if sufficient data is made available) 
tp quantify the cost impacts to the extent NASA’s historical data 
allows (not necessarily PER level) and to identify shortcommings of 
NASA’s historical record keeping practices. ■ (industry legally can and 
will make more meaningful data available for NASA use than for a 
potential competitor.) 

LMSC recommends NASA follow up on this PER Survey by implementing an 
Aerospace Executive Level Survey on Low Cost Program Practices . The 
main thrust of this survey would be to: 

o Determine if wide spread agreement or disagreement exists between 
the Vice President and Center Director levels and the levels 
of the working level manager" who, in responding to this survey 
, furnished the answers,, which present the summary opinion of [13+18 

6l$ who believe individual program cost reductions of 25 to 
Z>QPjo could be obtained by implementing low cost practices, 
o Obtain an executive level "feel" for which practice areas 
(documentation, specifications, program definition, contract- 
ing practice, overhead practices, accounting practices, etc.) 
could be reduced without significant risk impacts if NASA's 
(and maybe DOD's) practices were altered/ streamlined to 
eliminate unnecessary/excessive requirements and procedures. 
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o Obtain their personal "feel" for how much could be saved 
in each practice area. 

Results of this survey, results of the LCPP Study to date and the 
results of low cost program/ management practice studies such as the 
1974 EOS study are recommended as inputs for generating an Orientation 
Brochure and Survey Questionnaire if recommendation P is implemented. 
Implementation of an educational/promotional effort is recommended to 
assure NASA field centers/industry that NASA fully intends to buy 
more "new" space programs with dollars saved by implementing low cost 
practices , 

* These recommendations were made during the mid-term study review 
and are being acted upon by NASA. 
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1.0 General : As a task of the current Low Cost Program Practices Study, LMSC has 

re-writteri the specification MU- D- 1000; Engineering Drawings and Lists. This 
document .was subjected to detailed analysis prior to the actual re-write activity, 
to determine its impact upon aerospace programs, in terms of cost-driving consider- 
ations, and general applicability". The re-write task was approached on the basis of 
the outcomes of the detailed analysis forming the structure for the Low Cost vers- 
ion of the specification. So that NASA users of the Low Cost version of the specif- 
ication may be aware of the factors influencing the re-write, these factors, and 
the results of their study are given in the following text. 

2.0 Applicability of MIL- D- 1000 MIL-D-1000 has been the drawing specification 
invoked routinely for the majority of DoD and NASA contracts undertaken in the 
aerospace industry since 1957. The document, in its present form instructs contract- 
ors in the manner in which drawings prepared for the US Government are to be 
executed. In- itself , the document is neither long, nor highly detailed, yet is is 
an extremely complex document. The complexity arises from the fact that it invokes 
many other sub-tier specifications and standards, both military and industrial. 

The quantity of sub- tier specifications invoked is dependent upon the Form 
requirement imposed by MIL-D-1000, and one or more of ten categories of drawings 
that the government may require to be produced. The Forms cited in the document 
are 3 in number, and are as follows: 

■ o Form 1 requires: Full Military Compliance/Control 
o Form 2 requires: Partial Military Control 

o Form 3 requires: Drawings to Industrial Standards with Minimum 

Military Controls. 
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The categories, 10 in number are as follows: 

A. Design Evaluation Drawings 

B. Interface Control Drawings 

C. Service Test Drawings 

D. » Logistic Support Drawings 

. C. Procurement Drawings (Identical Items) 

E. Procurement (Interchangeable Items) Drawings 

F. Installation Drawings 

H. Maintenance Drawings 

I. Government Manufacture Drawings 

J. Interchangeability Drawings 

MIL-D-1000 defines all these categories, and the 3 Forms, and states that the Gov- 
ernment will define, and specify, per each procurement from a contractor; the 
Form(s) and category (ies) under which the drawings are to be prepared. 

Figure. 2.1. details the Document Dependency Listing for all tiers associated 
with MIL- D- 1000 and is given for information. 

For. space programs undertaken by both DoD and the NASA, the usual Form imposed 
has been Form 2, as experience with Form 1 has demonstrated that production to 
thp full requirements invoked is toe costly. Costs do not vary very much from cat- 
egory to category, and it is clearly evident from the analysis undertaken that 
the cost driving aspects of MIL-D-1000 lie in the Form invoked. 

In application of MIL-D-1000 to space programs, research indicates that the draw- 
ings producad under Government contracts do not vary very much in nature , whether 
Form 1, 2, or 3j is called out. The symbology used is much the same, and the 
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MIL D 1000 Sub-Tier Specifications 


■Federal Specs : 

ijU-P-00^61. Paper, Tracing. 

CCC-C-531 . .. Cloth, Tracing. 

I ’ ./ . . # 

Military Standards: , . 

MIL ST'D ICQa.V: EF10I NEERIKG DRAWING PRACTICES (Cost Driver) 

MIL-STD-1U3 ... Order of precedence, specs, stds. 

MIL- STD-9 Screw Threads 

MIL-STD-12C . . . . Abbreviations 
MIL-STD-lii . . « . . Architectural Symbols 

MIL-STD-15 Electrical Symbols 

MIL-STD-17 ..»>•• Mechanical Symbols 
MIL-STD-lS, , . . . Structural Symbols 
MIL-STD-29 . . . . . Springs , Mechanical. 

MIL-STD-3U Optical Element's ' ~.‘~ 

MIL-STD-275 . • Printed Wiring 
MIL-STD-806 .... Logic Symbols 

Industrial Standards : 

USAS Bh6.1 -62 Surface texture, roughness, wavines's & lay 

USAS' YlU.5 -66 Dimensioning & tolerancing for engrg dwgs , 

USAS YlU- 15-66 Electrical & Electronic Diagrams 

USAS Y 32.2-67 Graphic Symbols for Elec & Electronic _ diagrams 

USAS Y32. 16-65 Electrical & Electronic Reference Designations 

AWS STD A2.0-58 Welding Symbols 

AWS STD A2. 2-58 Non Destructive Testing -Welds. 


Form No. 

(2)(3) 

( 2 ) 


parts 
(2, only) 


Mil-Specs : 

mil-d-5630 

MIL-Q ! -9858a. . . . 

MIL-M -9868 

MIL-I-1:5208.... 
MIL-c-[l5662 . . . • 


Data, Engrg & Technical, reproduction of: 
Quality Program Requirements 
Microfilming Engrg Dwgs 
Inspection System Requirements 
Calibration of Measurement & Test Equipment 


( 2 ) 

( 2 ) ( 3 )• 
*( 2 ) 
(2)(3) 
( 2 ) 


MIL HDBKS : 


Federal- Supply Code; Name to Code 

Fi|-2 Federal Supply Code; Code to Name 

H6-1. Federal Items Identification Guide 

........ Mfrs Part & Dwg Numbering System 

Mil Manuals : ■ 

5220. 22M DoD Industrial Security Manual ^ _ (2)(3) 

U120.3M. Standardization; Policies, Procedures k Instructions. 


All the documents listed above are invoked by imposition of MIL- D-1000. Form -l t _ 

* ' (2) denotes Form 2 only. 

'(' 3 ) denotes Form 3 onlyT" 
-» Only if Requ'd by CERL 


?8-S«2 .1. 
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differences arise from the degree of assurance . applied .to certify compliance 
with the drawing requirements invoked. One of the difficulties arising from app 


lication of MIL-D-1000 to aerospace programs emerged clearly from this analysis, 
and can be described as the lack of understanding by technical personnel at 
contractors* plants, of the entire MIL-D-1000 and its sub-tiers assemblage of 


documents. A similar lack of understanding was exhibited by Government employees 


with whom the subject was discussed. Such a lack of understanding is scarcely 
surprising in the light of the fact that should Form 1 be invoked 38 sub-tier 


documents are involved, for Form 2 the number drops to 12 T and for Form 3 only 
k sub-tier documents are involved. 


In considering the applicability of MIL-D-1000 to space programs using 
either conventional , or standardised hardware components, or both; the question 
arose: "Why not use Form 3, as the least costly, and least difficult Form to 
apply". The answer appears to be that the NASA would not be able to differentiate 
as to costs between Form 2, and Form 3, and would in all probability, be paying 
5brm_2 prices, for. Form 3 drawings. This condition would arise from the fact 
that adrospace contractors are accustomed to work to Form. 2, and the majority of 
nrafting Practice Manuals (DPM) have been structured about Form 2 requirements, 
and the major sub-tier document MIL-STD-100A, Engineering Drawing Practices. 

Even Form 3 costs are augmented by the mandate that compliance with MIL-Q-9858A 
is contractually binding upon the contractor. 

The conclusion was reached that the MIL-D-1000 document with its sub- 
txers is neither a Least-Cost promoting specification, nor is it directly appl- 
icabie to NASA needs for space programs. This conclusion prompted the LMSC 
rewrite of the document. 


w 
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3*0 Open Ended Specification Requirements & Practical Requirement Limitations : 
The present MIL-D-1000 specification tends to be open ended. In effect, unless 
a contractor possesses all of the sub-tier documents cited in the prime 
specification he could not produce an acceptable drawing. MIL-STD-100A, 
the major sub-tier document defines, with examples, how a drawing is to be 
produced, titled, lettered, bordered, detailed, etc., etc., etc. In turn, 
MIL-STD-100A, calls out those regulating documents which the DoD believes essen- 
tial for the reproduction, microfilming, preservation, packaging, shipping, etc, 
of drawings produced for the Government. MIL-D-1000 merely 1 categorizes the 
drawings by classification of end-use, and specifies the rigor of the control 
measures to be applied to the various levels as defined. Whenever the Government 
so elects, drawing measures can be augmented by special provisions within the 
RFP, and these must be observed by a contractor, if he is no"t be be adjudicated 
non-compliant . 

In terms of practical requirement limitations , LMSC is of the opinion that 
requirements should be made a simple as possible if costs are to be maintained 
at sane minimum level. Further, all pertinent information concerning drawings 
should be included in one document, as far as possible. Finally, assurance 
functions undertaken to determine compliance with specified requirements should 
be maintained at an irreducible minimum, to obviate "checkers for the checkers". 
These factors have been recognized, and treated in the re-write of MIL-D-1000. 
U.O Requirements vs State of the art Relationships : As stated in paragraph 3.0, 

MIL-D-1000 does not specify methodology for the preparation of drawings. 

It cites instead MIL-STD-100A for that purpose. This Standard, while elaborate- 
ly detailed, and inclusive of many samples of drawing requirements for letter- 
ing, et alj is outmoded in the light of current aerospace requirements. 
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The document does not contain information pertinent to electronic microcircuits, 
nor are references made to computer produced drawings, or other mechanical aids 
to drafting such as "Paste-Up" superimposable symbols, letters, and other forms 
of decals. All of these data are included as part of the revised specification 
generated to be a Low Cost replacement for MIL-D-1000. 

Functional Division of Contract Labor : The production of drawings and lists 

compliant with MIL-D-1000 involves the following segments of contractor direct 
labor: " * ■ • - 

o Design Engine ering/De sign Drafting 
o Quality Assurance, Engineering and Inspection, 
o Configuration Management, and Configuration Control. 

Research into several NASA programs indicated that costs of direct labor in 
these three areas can be appreciable in the first two. The cost impact on 
drawing production of Configuration Management, does not appear to be great, 
at least on the two programs considered, MVM-73 and AE. The details of these 
impacts, on direct labor costs were the subject of two previous EMs to which 
reference should be made for further information on the subject. These EMs 
are : 

•LCPP-2. Impact of MIL D 1000 Upon Space Programs, and Proposed Cost 
Improvements of Changes. Jan 7/7g 

LCPP-3. Potential Cost Benefits Derivable From Changes to 10 Selected 
Cost Driving Specifications. Jan lit/7g . 

LCPP-2 includes a summary chart which is repeated in this EM to illustrate ,( Fig 3.1) 
the program costs involved in making drawings to Forms 1,2,&3, and the high con- 
trbution of Quality assurance to these costs. In addition, if it is assumed 
that Form 3 drawings can be produced for some unit cost, then: 

Form 3 Drawing Costs = 1, Form 2 Drawing Costs = l.b approx.. Form 1 = 1.73 v 


TYPICAL DRAWING COSTS FOR Ml L-D-1000 FORMS 
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As has been stated previously in Pargraph 2.0 of this EM, Form 3 costs for 
the production of drawings are not Least Costs, as quality provisions are still 
required, and most DPMs in use by contractors are modelled after Form 2, and 
even Form 1 requirements to some extent. The LMSC re-write is oriented toward 
Least-Cost Practices, and treats methods of lowering the cost below those app- 
licable to Form 3. 

6.0 Specification and Requirement Overlaps: IMSC has found, that in the course 

of studying specifications, there are numerous' documents whose requirements 
overlap. An example would be MIL-Q-9858A Quality Assurance Provisions, which 
to an extent overlaps the requirements set forth in MIL-I-U5208 Inspection 
Systems For Aerospace Contractors. In the case of MIL-D-lOOOthere are no over - 
lapping requirements contained in other specifications, since MIL-D-1000 deals 
with the general requirement for the preparation of drawings for- delivery to 
the Government, whereas MIL-STD-100A and others deal with Engineering Drawing 
Practices . 

7*0 Contractors Probable Response to RFPs and Associated Specifications Lists: 

To combat response by contractors to the Form 3 requirements of MIL-D-1000 
with Form 2 level costs, LMSC believes that the following measures must be 
instituted by NASA. 

1. Impose the rewritten version of the drawing specification. 

2. Be highly specific as to exactly what is expected of the contractor(s) 
using the RFP and other contractual documents as vehicles for this 
purpose . Any deviations from the drawing specification should be 
detailed in these documents as applicable . 

3. Ensure that contractor's ) DPMs be revised to reflect Low Cost 
Practices, before the DPMs are approved, and make Such approval 
a binding condition of each contract . 


LCPP-13 

Page 9 of IQ 


h. Assure that drawing costs are monitored per contract, and 
that such costs are isolated, as a cost entity separate from 
the overall costs of documentation. 

5. Coordinate the Low Cost aspects of the rewritten specification 
replacing MIL- D- 1000, with the NASA & Tri-Service Specification 
, Review Board. This measure will facilitate any cost saving measures 

recommended by the DoD being incorporated into the final drawing 
document. 

8.0 General Specification Changes ; At the outset of the specifications analysis 
task there was agreement between NASA/HQ and LMSC that a listing of the changes 
recommended to any specification under study would be made. This listing would 
serve as a basis of comparison for the specification under study, as written, 
versus the cost saving aspects of the changes. An example of this technique 
was the analysis and evaluation of NHB.5300.i*(iB) Quality Assurance Provisions; 
which formed the subject matter of EM LCPP-6 issued February 2k/l$. 

In the case of MIL- D- 1000 changes to the document structure, and provisions/ 
requirements were of such a major nature and extent, that no such list could be 
considered practical. LMSC undertook therefore to generate a completely new 
specification, the draft of which was submitted to NASA August 20/73? for re- 
view and comments. 

9.0. Cost Effectiveness of Changing and/or Eliminating Sub-tier Specifications . 
Figure 2.1 shows the specifications associated with MIL-D-1000 for the three 
Forms. The rewite of the drawings specification preparated by IMSC eliminates 
all sub-tier specifications previously invoked by MIL-D-1000, and uses only 
U documents in support of the new specification. Of these, three (3) are used as 
references and guides; viz: MIL-STD-100A where necessary, MIL-M-9868 Microfilm- 
ing, and MIL-D-58i;0 Preparation of Data for shipment to the Governement, also 
only where necessary. The only document imposed is 3220. 22M, the DoD Industrial 
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Security Manual, which sets forth the measures’ to be taken to protect the 
Government's rights to security. This document too, would only be used in 
cases where classified drawings were to be produced. 

The new drawing specification is believed to be Cost Effective since: 
o All sub-tier documents save those cited above have been eliminated 
as mandatory inclusions, and those cited are to be used as references 
and guides. 

o Document acquisition and reading time have thus been greatly reduced, 
o The tracking oi specification changes, and their incorporation into 
the specifications to which these changes relate, has been reduced, 
by the elimination of the sub-tier documents, 
o Confusion caused by the proliferation of the number of inter-related 
specifications, has been reduced by greatly reducing the number of such 
documents, which in turn lessens the chance of document misinterpretation. 
All these factors contribute to saving time expended by technical personnel, 
and LMSC studies have demonstrated that the greatest cost to NASA is "People 
Dollars" both within the NASA organization, and at contractors' facilities. 
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1.0 General : As a task of the current Low Cost Program Practices Study, IMSC 

subjected MIL-D-1000 Engineering Drawings & Lists to intensive analysis to det- 
ermine its cost affecting consequences when applied to NASA space programs. 

The outcomes of this analysis, documented in EM LCPP-13 Specification Analysis 
MIL-D-1000, were (a) a decision' to re-write the specification, and (b) furnish 
an implementation commentary on the new specification, to facilitate its applic- 
ation and use. 

2.0 List of Changes : A list of changes is as follows: 

(1) Forms 1,2 &3 of the MIL-D-1000 specification were eliminated, the 
new document requiring a level of control even less than that of 
the old Form 3. 

(2) The number of categories was reduced from 10 in the MIL-D-1000 
to 3 only in the new one. New categories are: 

o Design Descriptive/disclosive 
o Procurement 

o Installation & Maintenance. 

(3) Every MIL-STD and MIL-Spec was eliminated from the sub-tier of 
MIL-D-1000 documents except one. This one 3220. 22M was retained in 
order to protect the Governments rights to security. The document 
is the Industrial Security Manual. 

(10 MIL-STD- 100A Engineering Drawing Practice, and its associated Indus- 
trial standards were cited as references and guides only , to be used 
in those cases where contractors' Drafting Practices Manuals (DPMs) 
are not considered adequate by NASA. 
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. ..(5). Contractors' DPMs were cited as the regulating documents 
for drawing production, subsequent to approval by NASA. 

/.hese are the major changes, the minor changes are contained in the detailed 

re-write of MIL-D-1000 which is tentatively titled A low Cost Engineering Drawings 
and Lists Specification . 

3*° Potenti al Implementation and Interface Problems : At the outset of the specif- 
ication task there was some concern that the adoption of a new, and considerably 
simplified drawing specification might cause some implementation problems. So 
far as may be determined currently, a problem might arise from contractors 
charging MSA Form 2 prices for drawings produced to the requirements of the new 
.specification. This problem is discussed at length in both EM LCPP-13 aforement- 
ioned, and the new specification itself, together with the measures to be observ- 
ed in preventing such a condition. The condition is believed to be of a temporary 
nature, and will decrease, or be- eliminated as contractors become familiar with 
working to the new document. No other problems of interface/implementation are 
forseen. 

Scope of Sayings Anticipated When New Document is Implemented : During the 
course of the LCPP Study, cost savings to be derived from, re-write and simplific- 
ation of MIL-D-1000 were studied extensively. Results of the study were document- 
ed in depth in the following EMs: 

o LCPP-2 Impact of MIL-D-1000 Upon Space Programs and Proposed Cost 
Improvements of Changes . Jan 7.1975 

o LCPP-3 Potential Cost Benefits Derivable, from Changes to 10 Selected 
Cost Driving Specifications . Jan 1U.1975 
o LCPP-13 Specification Analysis, MIL-D-1000 August 26.1975 
o The rewritten specification itself. 

A short summary of the results may be given as follows: 
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Assigning soma nominal cost to a program associatad with production of 
drawings to the f*rmj requirements of MIL-D-1000, then the same drawings 
produced to F orm 8 would cost approx. liOji more , and drawings to Form 1 
requirements would cost approx. 75 % more. 

Conversely, Fom 3 represents a savings of 57$ of program costs for drawings 
vs Form 1. These, and other drawing cost data are given in the referenced EMs. 

In addition to the cost savings afforded by adoption of Form 3, LMSC believes 
that further cost benefits may accrue' from adoption- of the re-written Low Cost 
Drawing specification. Currently, there is no precedent for such a statement, 
as there has been no program exposure to the re-written specification, but 
discussion with X»0 SSD Chief ttaaft^an, and LMSC manufacturing representatives 
has elicited the opinion that a savings ranging from Sg8$ of the costs incurred 
m producing drawings and lists to Form 3 requirements may be expected. This op- 
inion 'is based largely on the use of sketches, and pencilled lines only in the 
H 4 D engineering phases, and the minimisation of drawing changes. Such savings 
might not accrue immediately following adoption of the new specification, as them 

may be .minor costs in simplifying contractors' DPMs, and re-orienting drafting 
personnel. 

5.0 Ris k ., Considerations : In terms of the classic definitions of Program Risk; 

i.e. Risk to Hardware Perfonnance, Risk to Maintaining Program Schedule, and 
Risk of Incurring Additional Program Cost, LMSC forsees no risk increment in 
any of these areas. Since IMSC believes that personnel drafting time will be 
caved by imposing the new drawing specification on programs, risk to schedule, 
and program cost should. decrease, if anything. 
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6.0 New Drawing Specification & Contractors 1 Low Cost Responses : 

The Low 'Cost* Drawing Specification, in' its present form, contains rationale 
for the changes made to improve costs beyond the Form 3 mandates of MIL-D-1000. 

A background of the history of the MIL-D— 1000 document also is included for 
those unfamiliar with the document, and its requirements. The new specification 
is arranged in such a manner, that all of this rationale and history can be 
omitted without disturbing the actual textual matter of the specification. 

Should. NASA elect to issue the specification with. the rationale deleted, the 
document would appear to be relatively innocuous. In this case, NASA should 
not only cite the new specification in the contractual section of an RFP, but 
also require responding contractors to comply with the specifcation, and in 
addition, require the contractors to include in their proposal any cost saving 
measures they deem applicable, over and above those required by the specification . 
With any new specification, there is a period during which its impact is not 
f ully understood and accepted for what it purports to accomplish. By means of the 
RFP, ocntractors can be encouraged to think Low Cost, and required to furnish, 
cost saving methods, and procedures. 

Should NASA elect to furnish the Low Cost Drawing Specification to con- 
tractors, and impose it in its entirety, a precedent will be set. Currently, 
specifications do not contain the "Reasons Why" what is specified is included in 
the text of the documents. As such, they are often sterile, treating only "Dos 
and Don'ts". If reasons are given for some of the specific requirements, more 
visibility is afforded to the contractor, allowing him to use his expertise in 
making cost effective decisions. 

LMSC emphasizes strongly the need to use the RFP as an instrument to 
convey to the contractor exactly what is needed as early in a program as possible . 
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Under Contract NAS W-2752 to NASA/HQ Low Cost Systems Office, LMSC is performing 
a Low Cost Program Practices Study. Previous tasks in the study had identified 
certain low-cost practices which, if implemented by NASA, could result in signi- 
ficant savings in contractor costs. 

In preliminary review of the low-cost practices with NASA Program Managers, some 
evidenced a feeling that implementation of the practices would; increase the program 
risks. This task, involving the qualitative evaluation of risk impact for each 
identified low-cost practice, was therefore performed by LMSC to provide additional 
basis for NASA decisions on whether to implement the practices in general and/or 
on specific programs. 


The basic subtasks which comprised the total Risk Impact task as described in the 
LMSC Technical Proposal, LMSC-DU 26636 , par. 5. 2. 6. 2, and as modified by coordina- 
with the Low Cost Systems Off i os, were: 

(a) Establish a list of Low-Cost Practices with brief definition 

(b) Catalog each practice, qualitatively , as "increased","no-change", or "decreased 
risk (compared to historical practice which is replaced) 

(c) Develop and explain the rationale for the ri3k assignment to each low-cost 
practice 

(d) Identify the type of data required/desired to allow later quantification of 
the risk impact. Outline a- plan for (l) tracking of cost and risk impacts 
as a result of implementing low-cost practices and (2) acquiring data 
necessary to quantify the cost and risk impacts. 

This engineering memo presents the data on the risk assessment analysis, combining 
items (a), (b), and (c) above. Work on item (a) is continuing. 

For ease of reference, all of the data have been formatted on a set of Data Sheets, 
each sheet covering a specific low-cost practice. 
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GLOSSARY OF TERMS 

Cost Impact - The cost impact is the cost-reduction effect which implementation 
of the low-cost practice has upon contractor program costs. The terms "High", 
"Moderate”, and "Low” as used on the data sheets herein are relative o nl y and 
indicate the general degree of cost-driver characteristic for each named 
practice. For example, a "High" rating of cost impact for Engineering 
indicates that contractor average engineering costs can be reduced signifi- 
cantly if the practice were applied to a program (s ) . 

: j : ' . i . . i 

Component - A component as used herein is an assembly or a black box ; examples : 
Star Tracker, Transmitter, Valve Assembly, Tape Recorder. 

- A part as used herein refers to a piece-part; examples: transistor, 
bearing . 

Subsystem - A group or set of components with interconnects of electrical cabling 
or piping and representing a functional portion of the total spacecraft; 
examples: Communications Subsystem (comprising transmitters, receivers, anten- 
na, etc.). Electrical Subsystem (comprising batteries, solar arrays, power 
control unit, e tc . ) . 

Risk, Delta - The delta risk designated on the data sheets represents the pot- 
ential change in risk (from the risk inherent with the historical practice) 
if the alternate low-cost practice is implemented. Example: a "decrease" 
risk designator for a practice under "Performance” indicates that implement- 
ation of the low-cost practice will decrease the uncertainty that the planned 
mission performance will be attained. 

Risk, Cost - Cost risk is the general probability that costs will grow beyond those 
estimated or targeted for the program; increased cost risk represents increased 
probability that the program costs will overrun. 

Risk, Performance - Performance risk is the general probability that the planned 
hardware mission performance will not be attained. 

Risk, Schedule - Schedule risk is the general probability that the planned 
schedule will be exceeded or deviated -from in a manner which will have an 
influence on increasing cost or not meeting a critical milestone. For 
example, if the hardware completion date is keyed to a single available 
launch window, any increase in schedule span (non -recoverable) can be critical 
to the program. 
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ATTACHMENTS - Risk Impact Data Sheets 

LCP No, 1-1 Emphasize Complete Definition of Program Elements in Phase C/D RFP 
Preliminary Review of RFP by Contractor 
Use RFP to Encourage Lower-Cost Response 
Streamline Response Data Required by RFP 

Emphasize Minimum Acceptable Mission/System Performance Reqts. in RFP 
Lower-Cost Alternate Proposals 

Reduce Requirements for Contractor Plans in RFP's 


1-1 

1-2 

1-3 

1-U 

1-5 

1-6 

1-7 


DESIGN 

2-1 Minimum-Cost Design Requirments for Spacecraft and Experiments 

2-2 NASA Low-Cost Design Handbook 

2-3 Use Flight-Proven Hardware 

2-4 Use of NASA Standard Components 

2-^5 Lower-Cost Design Requirements for Subcontractor/Supplier Equipment 

2-6 Higher Design Margins -Less Testing 

2-7 Lower-Leval Requirements for Secondary Experiments 

2-8 , Standardized and Common-Usage GSE 


DOCUMENTATION 

3-1 Use Low-Cost-Driving CDRL's 

3-2 Documentation-Need Evaluation Form 

3-3 Establish a Central Documentation Control Function 

3-4 Use Direct Contact in Lieu of Paperwork 

3-5 Use of Contractor Internal Documentation 

3-6 Lower-Cost Engineering Drawings for NASA Unmanned Space Programs 

3- 7 Data Bank to Identify Documentation Costs and Cost Reduction Areas 

MANUFACTURING 

4- 1 Increased Use of Simplified Tooling 

4- 2 Use of Contractor/Subcontractor Proven Manufacturing Processes 
PROGRAM DEFINITION AND CHANGES 

5- 1 Emphasi z e Earlier and More Complete Program Definition 

5-2 Program Definition Assessment Check List 

5-3 Establish Minimum Change Policy During Phase C/D 

5-4 Provide Incentives for Minimizing Cost-Increase Changes 

5- 5 Low-Cost Program Practice Check List 

PROGRAM MANAGEMENT 

6- 1 Low-Cost-Oriented Program Staff 

6-2 New Set of Low-Cost CER's 

6-3 Small Program Teams and Minimum Interfacing 
6-4 Visibility of Program Practice Cost Impact 
6-5 Standardized Cost/Schedule Control Systems 
6-6 Standard WBS and Definitions for Cost Elements 
6-7 Common Cost Data Bank (Cost Records) 

6-8 Experiment Interface Improvement 

6-9 Reduce Program Hardware Traceability Requirements 
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ATTACHMENTS - Risk Impact Data Sheets (Continued) 

QUALITY ASSURANCE 

LCP No. 7-1 Rewrite of QA Specification 

7-2 Use of Contractor Quality Manual -Low Cost Version 
7-3 Lower-Cost Material Review 

7-li Reduce QA Effort on Previously-Qualified Hardware 

7- 5 Reduce Quantity of Inspection Points 

RELIABILITY, MAINTAINABILITY. SAFETY 

8- 1 (No data sheets; reserved for later additions ) 

SPECIFICATIONS 

9- 1 Revision or Rewrite of Current Specs to Low-Cost Versions 
9-2 Cost-Effectiveness Analysis of Spec Applications 

9-3 Do Not Apply General Specs to Contractors 
9-U Establish Central Spec Control and Coordination 
9-5' Use of a Reduced "Standard" Spec List 


TESTING 

10-1 Optimize Testing for Minimum Cost 

10-2 Test Software and Equipment Standardization 
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1. BACKGROUND 

jMany NASA personnel have expressed the feeling that application of low-cost 
practices (spending less and/or doing less on programs) would automatically in- 
crease program risks. Others in NASA and industry feel that many program prac- 
tices can be altered without impacting risks in any way. The major purpose of 
this comparatively small task is to provide the initial countering auguments to 
show which of the array of low-cost practices can be implemented at essentially 
no increase of risk to a program. 

Because historical risks had never been quantified, it was necessary to approach 
the low-cost practice risk comparison on a qualitative rather than on a quanti- 
tative basis. LMSC has focused the effort on providing a rationale for each 
low-cost practice, explaining why there is a probable increase, decrease, or 
no change in the program risk as compared to the historical practice. 

2. BASIC TASK APPROACH 

The steps in the analysis comprised! consolidate all of the potential low- 
cost practices into a single listing, categorizing them into groups, and pro- 
viding an assignment of risk potential as compared to the historical practice. 
Quantification of the cost impact and of the risk impact of low-cost practices 
is not considered practical at this time. Therefore, a qualitative analysis 
has been made and the risk delta designated. 


The practices listed herein are applicable to expendable-booster -launched 
spacecraft and can be applied to near-future NASA programs. However, all can 
be applied to the unmanned spacecraft of the Shuttle era also. Those additional 
low— cost practices which can result with use of the Shuttle have not been ad- 
dressed here. 

The low-cost practices listed are principally those which NASA can implement 
directly or indirectly in establishing program requirements which will allow 
reduction in contractor program responses and contractor costs. 

The areas of program risk that have been used for this initial analysis are 
Hardware Performance , Program Schedule , and Program Cost Overrun. 


It is anticipated that NASA will, after review and embellishment of the initial 
data! contained herein, prepare and issue separate implementation directives for 
all lor portions of the practices listed.. The basic intent, therefore, in this 
first effort was to establish the basic framework upon which a complete listing 
of low-cost practices could be built over a period of time and the risk concepts 
made a little more tangible. 
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3* LISTING OF LOW-COST PRACTICES 
3*1 Selected Categories 

For convenience of reference, the listed low-cost practices- have been 
grouped into 10 categories. Each Data Sheet (attachments to this engineer- 
ing memo) is identified with the category number and a dash number? for 
example, "LCP No. 3-1” is the first data sheet under Documentation category. 

The categories selected for this initial listing are: 

(1) RFP's (and Contracting Practices) 

(2) Design (Spacecraft, Experiments, GSE) 

(3) Documentation 
(It) Manufacturing 

(5) Program Definition and Changes 

(6) Program Management 

(7) Quality Assurance 

(8) Reliability, Maintainability, Safety 

(9) Specifications 

(10) Testing 

3 ‘ 2 limiting o f Practices Listed to Significant Cost-Drivers 

Because of the desire to emphasize high or moderate -level cost-driver 
practices, many of the lesser-impact practices were omitted from the listing. 

3.3 — 1 — - of NASA Practices Which Impact Contractor Costs 

By ap-eement with the Low Cost Systems Office, the practices included in 
the listing are those NASA, practices which impact the contractor program cost 
Although a number of "internal” contractor and internal NASA practices were 
reviewed, they were not included on the listing? actually, they are beyond the 
agreed-upon scope of the current LMSC study. It seems moreover that most of 
these internal practices are a secondary effect of implementing the principal 
NASA/contractor interface requirements; the internal practices will tend to 
change when the prime-driver NASA requirements are altered to low-cost. 

3.4 Design Practices 

Most of the practices listed in the "Design" category address the general 
aspects of the engineering approach but do not include all of the specific 
recommendations for low-cost design of spacecraf t/experiments/GSE . An expan- 
sion of the basic list, with embellishment, could be the base for a Low-Cost 
Design handbook? preparation of such a handbook is recommended on Data Sheet 
wo. LCP 2-2 as a new NASA low-cost practice implementation tool. 
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3.5 Impact of Reliability, Maintainability, Safety Practices 

Our preliminary listings of low-cost practices included several for this 
category. Subsequent review of specific practices indicated that the cost im- 
pact was not significant for any single practice on typical unmanned space 
programs. We feel that this category deserves continuing attention, however, 
and have maintained it in the listing for possible later addition of specific 
low-cost practices therein by NASA. 

3.6 Manufacturing Practices 

Because most of the manufacturing practices are internal to the contractor 
and not directly impacted by the basic NASA requirements, this category has only 
two Data Sheets therein. In general, review of several historical NASA hardware 
program RFP's indicates little specific attention to manufacturing requirements. 
It may be that there are no additional significant cost-driver practices which 
can be influenced by NASA requirements upon the contractor. Nevertheless, the 
category has been retained in the listing for possible later add-ons by NASA. 

3.7 Consolidation of Low-Cost Practices for Data Sheets 

A considerable amount of time was spent, in review of NASA and LMSC low-cost 
practice data and data from other sources. The principal data used in these 
analyses are identified in the "References" of this engineering memo. 

A consolidation of the various preliminary lists of low-cost practices was 
made, with emphasis upon including those which could be the nucleus of action 
items by NASA. Although some of the practices listed may be in the process of 
implementation in a specific NASA field center or by efforts of the Low Cost 
Systems Office in KASA/HQ, they were listed so that a broad scope of practices 
coverage could be represented. 

The LMSC preliminary low-cost practice listing were also reviewed for probable 
degree of cost impact and several were combined and some omitted from the final 
listing and data sheets included herein. The cost impact was qualitatively 
determined as "high", "moderate", or "low". Where the practice impacted certain 
contractor functional areas more than others, it was so noted on the data sheet. 
In many cases, the cost impact was rated as applying (essentially equally) to 
"all areas'! 

3.8 Use of the Low-Cost Practice Listing 

The set of data sheets included herein can be a nucleus set of data which 
could accompany a NASA -prepared "checklist" of low-cost practices. This check- 
list could be provided to responsible program/project managers and others in 
NASA as an initial guide or instruction for implementation. If desired, the use 
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of such a checklist could be made mandatory, with approvals required by 
NASA /HQ for any low-cost practice which was not incorporated into the new 
program (following a pattern similar to that employed in implementing usage 
of the NASA Standard Components throughout all NASA programs). 


U. RISK IMPACT ANALYSIS 
U.l Types of Risk 

There have been many publications/documents released on the subject of risk. 
Most of- them deal with business risk, emphasizing the economic aspects of new- 
bttsiness ventures or the actuarial information used by insurance companies. 

These data are based upon firmly definable variants and/or a collection of 
historical data on a very large universe of samples over a long period of time. 
For risk assessment of small -quantity spacecraft programs, each essentially 
different from the other, the task of specifically identifying risk is much 
more difficult and intangible. 

For the purposes of this analysis, we have taken a qualitative approach in as- 
sessment of risk. The general premise applied is that risk characterizes 
the level of consequences of making a wrong decision/prediction . There appears 
to be fairly general agreement on the types of risk pertinent to aerospace 
programs; these have been used in the risk assessment and are in three categor- 
ies as follows : 

a. Performance Risk - This type of risk is the uncertainty that the techni- 
cal performance and reliability of the operating system will be as pre- 
dicted or as required. Any increase in the probability that the mission 
performance will be degraded or that critical failure will occur is 
classified as an increase in performance risk. 

b. Schedule Risk - This type of risk relates to the time interval between 
the start of the hardware development and the availability of the com- 
pleted space vehicle (or launch date). Assuming that all major schedule 
milestones are critical, schedule risk increase is the probability that 
one or more of the milestones will not be met or more importantly, that 
the planned launch date will be delayed, 

c. Cost Risk - Cost risk is the uncertainty that the actual costs of the 
program will be different from those estimated. In most cases the nega- 
tive condition is emphasized, wherein increased cost risk represents a 
higher probability that there will be a cost overrun on the program. 
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Risk for Unmanned Spacecraft With and Without ' --a kkir: ; : 

It should be re-emphasized here that the risk assest. made in this 
study, are pertinent to unmanned spacecraft programs and limited to expendable 
spacecraft launched by expendable launch vehicles. The scope of the study was 
limited in this manner to obtain risk impacts of low -cost practices which 
could be implemented now or in the very near future. 

The introduction of the Space Shuttle as the launch vehicle, coupled with its 
capability to retrieve and/or repair on-orbit those spacecraft which have failed, 
will lower the program risk for many low-cost practices. Also, use of the 
Shuttle will allow implementation of additional low-cost practices not included 
in the listing herein. An example is the potential reduction of spacecraft, 
subsystem, and component MTBF requirements for those spacecraft which are 
retrievaole by the Shuttle (Note: this practice was omitted from the list 
presented herein because there appealed to be no cost saving in reducing the 
operating life or increasing the flight failure probability if an additional 
spacecraft being launched were the only means to accomplish the mission objective). 
Conversely, using the Shuttle for retrieval and repair can be shown to less 

costly than designing the spacecraft for longer lifetime and higher levels of 
reliability. 

As an extension of this initial study, follow-on effort by NASA should be ac- 
complished looking at the additional low-cost practices and/or change in risk 
impact as a result of using the Shuttle. 

U.3 Risk Assessment 

J: 3 ' 1 ^iJ^ignator for Change in Risk - The basic risk of implementing each 
listed low-cost practice has been estimated in a qualitative manner. Broad 
categories of "increased risk", "no-change", and "decreased risk-" were estab- 
lished j each of these represents a delta change in risk when reolacing the 
historical practice with the listed lbw-cost practice. These designators were 
applied to each practice separately for each of the risk categories. 

Initially, it was planned to estimate the change in risk as high, moderate, or 
low. Specific considerations of the practices, some of which can be varied in 
degree of application, forced us to a conclusion that the high/moderate/low 
sub-classifier tion was not meaningful in the qualitative assessment. 

k,3m2 — — ts 0f the Risk Assessment - Each low-cost practice was reviewed 
separately for its impact on risk. Figure 1 is a tabulation of the risk desig- 
nators applied to each of the practices. 

The availability of the "decreased risk" designator proved quite useful. Of the 
50 low-cost practices analyzed, 27 were judged to offer decreased risk in one 
or more of the three risk categories; performance, schedule, and cost. 

10 

LOCKHEED MISSILES & SPACE COMPANY. INC 



lcpp-15 



Risk** 




Ri 

sk*» 

Z] 

LGP Number* 

• 

M 

U 

(2 

- * 

TJ 

V 

*6 
c n 

4» 

8 


LCP Number* 

• 

u 

9 

eu 

Sched 

Cost 

1-1 

NC 

D 

D 


5-1 

D 

D 

D 

1-2 

NO 

NC 

NC 


5-2 

NC 

NC 

NC 

1-3 

NC 

NC 

NC 


5-3 

NC 

D 

D 

1-li 

NC 

NC 

NC 


5-U 

NC 

D 

D 

1-3 

D 

D 

D 


5-5 

NC 

NC 

NC 

1-6 

NC 

NC 

NC 


6-1 

NC 

NC 

D 

1-7 

NC 

NC 

NC 


6-2 

NC 

NC 

© 

2-1 

D 

D 

D 


6-3 

NC 

NC 

NC 

2-2 

NC 

D 

D 


6-U 

NC 

NC 

D 

2-3 

D 

D 

D 


6-5 

NC 

NC 

NC 

2-k 

D 

D 

D 


6-6 

NC 

NC 

D 

2-3 

D 

D 

D 

■ . ■■■■■■■ . 

6-7 

NC 

NC 

D 

2-6 

D 

NC 

NC 


6-8 


D 

D 

2-7 

© 

D 

D 


6-9 

NC 

©. 

l§9 

2-8 

D 

D 

D 


7-1 

NC 

NC 

NC 

3-1 

NC 

D 

D 


7-2 

NC 

NC 

NC 

3-2 

NC 

NC 

NC 


7-3 

NC 

D 

D 

3-3 

NC 

NC 

D 


7-U 

NC 

NC 

NC 

3-U 

NC 

D 

D 


7-5 

NC 

NC 

NC 

3-3 

NC 

NC 

NC 


9-1 

NC 

NC 

NC 

3-6 

NC 

NC 

D 


9-2 

D 

NC 

D 

3-7 

NC 

NC 

NC 


9-3 

NC 

NC 

D 

U-l 

NC 

NC 

NC 


9-U 

NC 

NC 

NC 

U-2 

NC 

NC 

NC 


9-5 

NC 

NC 

NC 

* See list of 
on pages h,5 

LCP numbers and titles 

10-1 

NC 

D 

D 

"D"» Decrease; !'NC"» No Change; 
"I"* Increase 

10-2 

D 

D 

D 


Figure 1 - Risk Designators Assigned to Low-Cost Practices 
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A "no-change" designator was assigned for one or more risk categories on 38 of 
of the SO practices. 

Of large significance is that an "increased risk" classification was placed upon 
only 3 practices of the $0. This means that almost all of the low-cost prac tices 
listed on the 00 Data Sheets can be implemented with no increase in program risk . 
This was an unexpected result of the risk assessment performed. A partial ex- 
planation for the low incidence of increased risk practices may lie in our sift- 
ing out some of the lower-cost-driving practices which may have been designated 
as an increased risk] an example of this is the omission of a practice to lessen 
the type and amount of reliability analysis - this was omitted because the cost 
impact on the program was considered comparatively small. Had this practice been 
included, it would probably been designated as a minor increased risk in the 
performance category. 

4.3.3 Rationale for Risk Glassification Assignment - Each of the attached Data 
Sheets contains an explanation of the rationale or logic used when assigning 
the risk classification. 

The rationale have been made fairly brief, hopefully to assure that it is read 
by prospective implementors (program managers, etc.) of the low-cost practices. 
Each rationale statement was separately generated for the specific practice 
rather than being extracted from a "standard" list of rationale. 

5. RESULTS OF PRACTICE IMPLEMENTATION 

On each Data Sheet, there is provided an outline of the results of the low- 
cost practice implementation on a typical program. This information is an ex- 
tension of the practice description, discussing the principal program imoacts 
in terms of elimination/reduction of work items and reduction of cost in speci- 
fic areas or functions. 

Here again, highlights only of the program impact are defined, hopefully to the 
degree that the user of the Data Sheet can quickly obtain a feel for the objective 
of the practice and the extent of the cost reduction without having to read a 
longer and more general narrative. 
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1. LMSC-D387517 dated 30 May 197k> "Final Report - Low Cost Program Practices 
for Future NASA Programs", Contract NAS 8-28960 S/A #5 

2. IMSC-DU266U2 dated 18 June 1975* "Interim Results Presentation - Low Cost 
Program Practices Study (NAS W-275>2)" 

3. "Management Study of NASA Acquisition Process - Report of Steering Group", 
dated June 1971 (Richard McCurdy, Chairman) 

k. "AIA Study of Air Force RFP's" dated 9 January 1970, Administrative Memo No. 70-1 

3. "DOD's Space Test Program", Lt.Col. Frank a Paparozzi and Capt. Neal T. 

Anderson, Astronautics and Aeronautics , June 197k 

6. "Cost Growth in Major Weapon Systems", Dept, of Defense, by the Comptroller 
General of the United States, dated 23 March 1973 

7. "Recommendations for Develppment of Major Defense Systems and Solutions 

to Design Complexity and Cost Problems", National Security Industrial Assoc., 
dated October 1973 

8. "Cost Study of Unreasonable Contract Requirements" (Draft), Office of the 
Asst. Secy, of Defense (Installations and Logistics), Washington D.C., 
dated 30 November 1973 

9. "Lower Space Costs Mean More Space Flight", A.O. Tischler, Astronautics 
and Aeronautics , June 197k 

10. "Management Approach Recommendations", Report No. k, Earth Observatory 
Satellite System Definition Study, Contract NAS 5-20920, Grumman Aerospace 
Corp., Bethpage, N.Y., September 197b 

11. "Management Approach Recommendations", Final Report 1*, Earth Observatory 
Satellite System Definition Study, No. 22296-6001 -RU-03, TRW Systems Group 
Redondo Beach, Calif., 1 October 197k 

12. "Low-Cost Management. Approach and Recommendations", Report No. k, Earth 
Observatory Satellite Definition S t udy, No. 7kSDk2k3, Space Div. of General 
Electric, Valley Forge, Pa., 15 July 197k. 

13. "Historical Costs Panel Report to the NASA Space Cost Evaluation Program" 

Norman Rafel (Chairman), 20 March 1973 (NASA/HQ) ’ 

lk. "Space Program Payload Costs and Their Possible Reduction", NASA TM X-62223 
NASA/OAST, Edgar Van Vleck, Jerry Deerwester, Susan Norman, Larry Alton, * 
January 1973 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LCP No. : 1-1 2. CATEGORY : R EP's 

3. LOW COST PRACTICE ; Emphasize Complete De finition of Program Elements in Phase 

. -c/d rfp IIIZZIZZZZIIZZIZ 

Emphasize complete program definition based upon known technology. De- 
emphasize R&D flexibility in the RFP's for hardware production. 

3. 2 Based upon very thorough Phase A/B configuration definition and completion of 
tradeoffs, define all_ program elements and exactly what hardware is required. 


U. RESULT OF IMPLEMENTATION : 

U.l Eliminates costing omissions due to incomplete listing /identification of all pro- 
gram elements. “ 

4. 2 Cost reduction by elimination of "add on" changes. 

4. 3 Historical records indicate cost increases (due to changes) of up to 60% occurred 
during Phase C; an additional 37% during Phase D. Much of this increase was due 
to lack of complete program definition and could be eliminated by this practice. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering H] Testing Reliability Q 

Manufacturing!!!] Materiel!!] Qual.Assur. !!] 

6. DELTA RISK (increase, No Change, Decrease) 

Performance No Change Schedule Decrease 

7. RATIONALE FOR RISK ASSESSMENT ; 

7.1 Establishment and accomplishment of required performance level is independent 
of the specific time of definition: therefore no impact of practice on performance 
risk. 

7. 2 Schedules based upon complete definitions and known technology will require mini- 
mal changes, thereby decreasing schedule risk. 

7. 3 Cost risks will be decreased because more accurate cost estimates can be made 
when all program elements are completely defined. 


A-l 


All Other 


U 


.□ 


Cost Decrease 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 
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Page 1 o f 1 


2. CATEGORY: 


3. LOW COST PRACTICE : Prelimlriary Review of RFF by Contractor 


3.1 Provide preliminary RFP's to participating contractors for review and com- 
ments, particularly in regard to inclusion of low-cost practices; for example, 
change or reduction to the specification list to allow use of alternate specs/ 
standards for reduced cost. 


ii. RESULT OF IMPLEMENTATION ; 

U.l Improves probability of incorporation of low>-cost approaches into a program 
at earliest possible time. * “ 

4.2 Eliminates need for traditional "high cost" full- response proposal; contractor 
would submit only a "low cost" program proposal in response to low-cost RFP, 

4.3 Takes advantage of industry expertise in lower-cost methods and allows NASA 
pre- evaluation of same. 


3 , AREA AFFECTED & DEGREE OF COST IMPACT : (High* Moderate, Low) 

Engineering □ Testing □ Reliability □ All Areas 

ManufacturingCH MaterielC Qual.Assur. □ 


6. DELTA RISK (Increase, No Change, Decrease) 

Performance No Change Schedule No Change Cost No Change 

7. RATIONALE FOR RISK ASSESSMENT : 

7*1 The degree of performance risk varies basically as the level of performance is 
changed, usually the higher risks being associated with the higher performance 
requirements. The performance risk’ will be traded off by NASA/Contractor for 
each low-cost alternate approach recommended; approaches which increase risk 
will probably not be proposed nor approved. Therefore the impact of this prac- 
tice alone will not change risk. 

7. 2 The same rationale can be applied to the schedule and cost-overrun risks. 
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1. LCP No, : 1-3 2. CATEGORY; RPFIs 

3 LOW COST PRACTICE: Use RFP to Encourage Lower-Cost Responses 

3.1 Phase B and Phase C/D RFP's should clearly state: 

(a) Low Cost practices are required. 

(b) Practices in all areas should be evaluated. 

(c) Which areas should receive special attention. 

3. 2 Also, the RFP should indicate to what extent low-cost responses will be accept 
able in the proposal without risk to contractor of being ruled "non- responsive", 

3. 3 Require contractors to document cost tradeoffs of alternate low-cost practices 

during Phase B studies, including risk assessment and cost-savings estimates 
(documenting assumptions and estimating factors). 

з. 4 Require contractor during Phase C/D proposal to refine/expand results of 

Phase B low-cost practice studies to meet specific requirements of Phase C/D 
RFP. 

и. RESULT OF IMPLEMENTATION : 

U.l Obtain recommendations for program cost reductions, with risk assessment 
and cost estimates, which can be implemented in Phase C/D. 

4. 2 Obtain lower cost responses and estimates from contractors (by eliminating 

fear of being non- responsive). 


5 . AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering [!!| Testing [^] 

Reliability [![] 

All Areas | gj 

Manufacturing!!!] Materiell — 1 

Qual.Assur. CZ1 

■ n 

DELTA RISK (Increase, No Change 

, Decrease) 


Performance N° Change 

Schedule No Change 

Cost No Change 

RATIONALE FOR RISK ASSESSMENT; 


7.1 Program risks are not impacted by this general practice. An individual risk 
assessment (impacts on performance, schedule, cost overrun) will be 
required for each low cost approach proposed by the contractor and will be 
approved or disapproved by NASA. 


Note; See LGP No. 1-6 for condition where a response to a baseline requirement 
is required in addition to an alternate proposal. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
1. LGP No, : 1-4 2. CATEGORY ; RFP's 

3. LOW COST PRACTICE : Streamline Response Data Required b y RFP 

^ R-educe /eliminate /simplify data required in contractor proposals: 

(a) Detailed information not directly related to program. 

(b) Detailed data that is non-essential to the source selection process. 

(c) Detailed functional plans prior to completion of program definition. 


k. RESULT OF IMPLEMENTATION : 

U.l Save contractor bid/proposal money by eliminating preparation of non-essential 
material. 

4. 2 Allow contractors more time in a proposal to improve the quality of truely 
essential data. 

4. 3 Save time and money during the NASA source selection process and possibly 
improve selection results. 


5..AP A AFFECTED & DEGREE; OF COST IMP ACT: (High, Moderate, Low) 

Engineering □ Testing [J Reliability []] All areas [m[ 

Manufacturing!!] Materiel!!] Qual.Assur. C] Q 

6. DELTA RIS K (increase, No Change, Decrease) 

Performance No change Schedule No Change Cos t No Change 

7. RATIONALE FOR risk ASSESSMENT : 

7.1 Although this practice would reduce costs, it is upstream of the actual program 
and has no impact on program risks. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1 . LCP No, : 1-5 2. CATEGORY : RFP ' S 

3. LOW COST PRACTICE: Emphasize Minimum Acceptable Mission/System Per - 

iormance R equirements in RFP 

Categorize performance requirements as mandatory or desirable in Phase B 
RFP's: 

(a) Mandatory: Minimum capability to perform the mission at all. 

(b) Desirable: Higher level capability selected for the particular program. 

3 . 2 Require tradeoff studies of cost and risk versus performance during the Phase B 
studies with emphasis on using minimum- acceptable requirements. 

3 . 3 NASA, based upon Phase B study results and the cost/risk impacts, select and 

establish firm set of performance requirements for inclusion in Phase C/D RFP. 


u. RESULT OF IMPLEMENTATION : 

If this practice were rigorously and systematically imposed it would provide 
clearer visibility of the costs for meeting successively more difficult per- 
formance levels associated with mandatory versus desirable performance 
requirements and allow NASA to establish a firm and cost-effective set of 
performance requirements at a reasonable risk for inclusion in Phase C/D 
RFP. 


$. AREA AFFECTED & DEGREE OF COST IMPACT: (High, Moderate, Low) 


Engineering 


Testing I 1 Reliability □ All Other 


ManufacturingCZ] Materiell 1 Qual.Assur, □ 

6 . DELTA RISK (Increase, No Change, Decrease) 


m 

.□ 


Performance 


Decrease 


Schedule Decrease Gost Decrease 


7 . RATIONALE FOR RISK ASSESSMENT : 

7 i Would decrease performance risks experienced in historical programs by elimin- 
ating impossible or near impossible requirements based upon more detailed 
analysis cf what work tasks, state-of-the- art breakthroughs , processes, and 
costs are necessary to accomplish various levels of performance. 

7 . 2 Having a firmly-fixed set of performance requirements will allow better schedule 
planning and reduce schedule risk in Phase C/D. 

7 . 3 The tradeoffs to achieve cost-effective performance levels will provide a firmer 
base for program cost estimates, thereby decreasing cost overrun risk. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
!. LCP No. : 1-6 2. CATEGORY : RFP ' S 

3. LOW COST PRACTICE : Lower-Cost Alternate Proposals (Lower than Baseline) 

Request in the Phase B and C/D RFP that contractors submit alternate pro- 
posal for lower-cost alternate approaches, including altered performance 
requirements or schedules, which will result in significant cost reductions. 

3.2 Require a risk analysis to accompany each Phase C/D proposal, comparing 

the cost, schedule, and performance risks in complying with the RFP-outlined 
program versus the contractor-proposed alternatives. 

3. 3 Establish valid alternate lower-cost apprca ches as an element of contractor 
evaluation (source selection). 


h. RESULT OF IMPLEMENTATION : 

U.l If implemented, in both phases B and/or C/D, this practice will provide a. means 
of obtaining more contractors' ideas of how to achieve low-costs in programs 
and detailed evaluation of risks involved. This information can be obtained in 
two or more iterative steps before a Phase C/D contractor is selected. 

4.2 Will exercise industrial innovation and provide the base for implementing lower- 
cost programs (except for special proprietary methods, if claimed so by the 
responding contractors). 

4.3 If alternate low cost approaches are studied in Phase B, contractor Phase C/D 
proposal costs will not increase significantly. 

5. - AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering □ Testing Q Reliability Q All areas fu] 

Manufacturing!!] Materiel!] Qual.Assur. (Z1 Q 

6. DELTA RISK (increase , No Change, Decrease) 

Performance No change Schedule No .change Gost No change 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Although implementation of this practice provides data and estimating factors 
for comparing cost and risk impacts of both baseline and alternative practices, 
its overall impact on risk is conservatively judged to be no-change in all three 
risk categories. This is based upon the belief that low-cost alternatives which 
are shown to increase risks significantly will not be proposed by contractors 
normally nor will they be accepted by NASA for most flying hardware. 


Note : See LCP No , 1-3 for condition where baseline response is not required 

and a lower-cost proposal alone is desired, 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LCP No, : 1-7 2. CATEGORY : RFP 's 

3. LOW COST PRACTICE : Reduce Requirem ents for Contractor Plans In RFP's 

3-1 Do not require the preparation of 15 to 20 contractor plans (Mgt. , Eng'g. , 

Mfg, Test QA, Reliability, etc) to be included with each proposal response. 

(at a time when spacecraft system hardware is not yet completely defined). 

3.2 One alternative is to require contractors submit a set of general "standard 
pl^tts including relevant contractor internal operating procedures which, if 
validated by NASA, could be used in lieu of re -preparation of program- 
peculiar plans for each proposal. ~ ‘ 

3. 3 Another alternative: require only preliminary plan outlines sufficient to justify 
estimated costs in proposal. 

3*4 Require detailed final plans after contractor selection (only one per program). 

it. RESULT OF IMPLEMENTATION: 


U.l Validation of contractors' capabilities via a set of standard plans would avoid 
much of the expenditure now required to have several contractors elaborately 
develop and document program-peculiar plans at point when definition of program 
hardware/software items is incomplete. 

4.2 Would eliminate consideration of potentially incomplete and/or misleading plans 
from the contractor- selection process (save source selection costs). 

4. 3 Could save program costs by eliminating premature startup of the detailed plan- 
ning cycle. 


5. , AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering []] Testing []] Reliability | | AR Areas [m] 

Manufacturing!!] Materiel!!] Qual.Assur. D Q 

6. DELTA RISK (increase. No Change, Decrease) 

Performance No change Schedule No change Cost No change 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Delaying preparation of detailed program -peculiar plans until after all program 
elements are completely defined would not change true program risks. 

7.2 Premature (before program definition is complete) documentation of such plans 
may not uncover true risk factors and therefore be misleading in the source 
selection and program evaluation processes. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LCP No, : 2,-1 2. CATEGORY : Design 

3. LOW COST PRACTICE ; Minimum-Cost Design Requ irements for Spacecraft and 

^ ^ EypprirmpnKc; 

Place a minimum-cost general requirement upon spacecraft and experiments. 
Specify a mandatory-minimum level of hardware requirements to perform 
the minimum- required mission. 

3.2 Require mission performance /cost tradeoffs to show "choices" of performance 
levels under lower program cost ceilings. 


k. RESULT OF IMPLEMENTATION : 

U,1 Will reduce the cost of both spacecraft and experiments; will encourage: 

(a) Greater use of proven technologies or equipment. 

(b) Use of lower-cost materials /processes. 

4.2 Will force consideration of lower-cost methods of analysis and testing. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering 1 1 

Testing | | 

Reliability | | 

All Areas j' pjJ 

Manufacturing! 1 

MaterielCU 

Qual.Assur, f 1 

□ 


6, DELTA RISK (Increase, No Change, Decrease) 

Performance Decrease Schedule Decrease Q ost Decrease 

7. RATIONALE FOR RISK ASSESSMENT ; 

7.1 The easier attainment of lessened performance levels for new equipment will 
reduce the performance risk. 

7.2 The reduced development and qualification testing of lesser-performance equip- 
ment (average) will tend to reduce the schedule and cost-overrun risks. 
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LCPP-15 


1. LCP No.: 2-2 


DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE PageJL of -L 
2. CATEGORY: DESIGN . 


3. LOW COST PRACTICE : NASA Low-Cost Design Handboo k 

3.1 Prepare and issue a NASA-wide handbook (for 

de sidling low-cost spacecraft and experiments and GSE Hanabook to cover 
both expLdable-launched and Shuttle-launched/serviced unmanned spacecraft. 

, , T Tlc i ude all aspects of design which impact Mfg. , testing, arid operations; em- 
3 * methods, processes, standard components, stan- 

dard spacecraft, refurbishment /reuse of components. 


h. 1 RESULT OF IMPLEMENTATION : 

U.l General and widespread implementation of low-cost design techniques within 
NASA and throughout industry. 

4. 2 Callout of Handbook in new-program RFP's would insure early adoption and 
"payoff" in lower hardware /software costs. 


3. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering Q Testing H_ Reliability Q] Operations [h~] 

ManufacturingOEl Materiel Zi. Qual.Assur . C^ l 1-1 


6. DELTA RISK (increase. No Change, Decrease) 

Performance No change Schedule Decrease 


Cost Decrease 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Because the incorporation of low-cost design practices are not intended to alter 
program performance requirements, implementation will result m no change to 
performance risk. 

7 2 As a result of designs requiring less exotic and commercially available standard 
materials, simplified tooling and testing techniques and similar low-cost 
approaches; schedule and cost risks will be reduced. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LGP No, : 2-3 2. CATEGORIC : DESIGN 

3. LOW COST PRACTICE : Use Flight-Proven Hardware in New Spacecraft Design 

Where possible, require use of equipment developed and proven on previous 
program(s). 

3.2 Require minor modifications (where necessary) to existing designs rather 
than develop new hardware. 

(NOTE: Flight-proven hardware may or may not be listed as "available" in 
the "NASA "Catalog of Available and Standard Equipment". 


k. RESULT OF IMPLEMENTATION : 

U.l Reduction in engineering costs for new design and analysis. 

4.2 Minimum qualification testing; limited primarily to testing of interfaces with 
other vehicle subsystems, components. 

4. 3 Reduced documentation required for proven design. 

4.4 Mfg. processes already "wrung-out"; lessened QA required. 


5 « AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering |HJ Testing 
Manufacturing^] Materiel 


H 


M 


Reliability [~u] 

(T7 

Qual.Assur. 


6. DELTA RISK (Increase, No Change, Decrease) 


D 

□ 


Performance Decrease Schedule Decrease Cost Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Newly-developed equipment always entails some uncertainty in actual perfor- 
mance; use of pre-tested/flown hardware reduces this performance risk. 

7. 2 Re-procurement of the same hardware presents lesser schedule risks than 

establishing new mfg/testing procedures on a new-design piece of equipment. 

7. 3 The actual cost of proven equipment (with updates for inflation, etc.) offers 
a much firmer base than estimates for new equipment; potential cost overrun 
risk is reduced. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 


LCPP-15 

Page 1 of 1 


LCF No. : 2-4 2. CATEGORY ; DESIGN 

LCW COST PRACTICE: Use of NASA Standard Components (Equipment Items) 

3«1 Require the utilization of NASA Standard Components in implementing new 
spacecraft designs to avoid new development costs. 


U, RESULT OF IMPLEMENTATION : 

U.l Reduced design/analysis costs for components for each using program. 

4.2 Makes possible a quantity -buy approach which normally reduces the unit 
cost of components. 

4* 3 Provides a potential pool of equipment and larger flexibility for priority procure 
ment from selected sources. 

4.4 Negates need for component requal test and reduces required level of QA on 
components. 


5. - AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering [h] Testing [h] Reliability [ h] 

ManufacturingE] MaterielH Qual.Assur. ED 

6 * DELTA RIS K (Increase, No Change, Decrease) 


□ 

□ 


Performance Decrease Schedule Decrease Cost Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Equipment which has been selected as a NASA standard, has already undergone 
considerable testing and higher confidence of satisfactory operation exists. 
Performance risk is therefore decreased some from equivalent new program- 
peculiar component. 

7.2 Multi-program usage of a component will assure firmer source(s) for procure- 
ment (or from GFE pool), thereby reducing schedule risk. 


7. 3 With development /qual and initial procurement completed, actual cost histories 
are available and cost overrun risk is thereby decreased. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 


LCPP-15 

P&ge 1 of 1 


1. LGP No. : 2-5 2. CATEGORY; DESIGN 


3. LOW COST PRACTICE : Lower-Cost Design Requirements fo r Subcontractor/ 

3 1 Supplie r Equipment ' 

Require that NASA or prime contractors request from suppliers alternate 
proposals for equipment which will meet the performance /interface design 
requirements, but can be supplied at lower cost than hardware specified. 

3.2 In equipment specs, prescribe widest possible tolerances on inputs /outputs to 
allow consideration of lower- cost hardware which may operate at other than the 
nominal but within the allowed tolerance band. 


h. RESULT OF IMPLEMENTATION : 

U,1 Takes advantage of industrial experience in designing /producing low-cost 
competitive equipment. 

4. 2 Incorporation of lower-cost requirements into equipment specifications. 
4.3 Utilize available (and proven) industrial designs, mfg. methods. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering 0 Testing 0 Reliability 0 

Manufacturing^ Materiel 0 Qual.Assur. 0 

6. DELTA RISK (increase. No Change, Decrease) 


□ 

□ 


Performance Decrease Schedule Decrease Cost Decrease 

7, RATIONALE FOR RISK ASSESSMENT ; 

7.1 Use of existing supplier designs and techniques rather than establishing new 
equipment spec requirements will reduce the testing and performance risks 
and improve the probability of on- schedule delivery, 

7, 2 The use of proven mfg/test technology will also decrease the risk of cost overrun 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LGP No, : z ~ 6 2. CATEGORY : DESIGN 

3. LOW COST PRACTICE : Hi gher Design Margins - .Less Testing 

Require increased design margins on structure and equipment functional 
characteristics; affording less- sophisticated analyses and reducing or elimina- 
ting structural /functional testing. 


U. RESULT OF IMPLEMENTATION : 

U.l Design analyses can be simplified. 

4.2 Designs can be verified in many instances by analysis only, without requirement 
for test verification, or simplified testing can be used, 

4.3 Larger functional design margins decreases probability of special stresses or 
unforecasted failure modes and, in general, lengthens anticipated MTBF. 


$. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate) Low) 


Engineering [3 Testing 

Reliability [m] 

□ 

ManufacturingGI] Materiel 1 l! 

Qual.Assur, CD 

n 

DELTA RISK (Increase, No Change 

, Decrease ) 


Performance Decrease 

Schedule No Change Cost 

No Change 

RATIONALE FOR RISK ASSESSMENT: 


7.1 Preflight verification of structural /functional characteristics will be determined 
using very conservative analyses and, where necessary, simplified over-design- 
limit testing. Performance risk should thereby be lessened. 

7.2 Although minor decreases may occur in schedule and cost overrun risks as 
a result of testing simplification, a "no change" has been recorded. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1, LGP No, .: z ~ 7 2. CATEGORY : DESIGN 

3. LOW COST PRACTICE: Lower Level Requirements for Secondary Experiments 

3.1 if there are multiple experiments planned for a mission, segregate the primary 
from the secondary, the latter being those from which degraded (or lack of) 
data are permissible without jeopardizing the basic mission objectives. 

3.2 Limit the application of Reliability, QA, and documentation requirements for 
these secondary instruments /experiments which are not critical to the prime 
mission. 


U. RESULT OF IMPLEMENTATION : 

I. i The selection of certain mission data as "desirable" but not critical to 
* mission success allows the reduction of reliability /quality and cost on the 
associated mission- equipment, and certain spacecraft support functions; 
cost reduction of these equipments will result. 


3. AREA AFFECTED & DEGREE OF COST IMPACT: (High, Moderate, Low) 


Engineering i 1 Testing 1 1 

Reliability ( j 

All Areas (Ml 

Manufacturing!!!] Materiel!!—] 

Qual.Assur, 1 1 

. □ 

6. DELTA RISK (increase. No Change 

, Decrease) 


Performance Increase 

Schedule Decrease 

Cost Decrease 


7, RATIONALE FOR RISK ASSESSMENT: 


7*1 There will be a minor increase in overall performance risk because of the poten- 
tial failure of the secondary experiment(s) . However, this will have no impact 
upon the primary mission accomplishment. 

7 2 There will be a small decrease in both schedule risk and cost-overrun risk result- 
ing from the less-stringent requirements applied to the secondary experiment equip- 
ment. 


A-lU 


LOCKHEED MISSILES & SPACE COMPANY 



DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 


LCPP-.J5 

Page 1 o f 1 

1. LCP No, i 2-8 2. CATEGORY : DESIGN 

3. LOW COST PRACTICE : Standardized and Common Usage GSE 

3«1 Standardize selected items of GSE for universal usage among ditierent 

NASA spacecraft programs: transporters, test sets, servicing carts, etc. 

3.2 Re-emphasize use of existing-design GSE as base if it fits the multi- application 

requirements; re-procure additional items to previous designs rather than redesign 
requalify. 

3. 3 Expand application of existing GFE inventory to future programs; utilize NASA 
"Equipment Visibility System (EVS)" as checklist for callout in RFPS. 


h. RESULT OF IMPLEMENTATION : 

U.l Standardization of GSE would also impact spacecraft design, providing the incen- 
tive for standard testing and pre-launch operations and standardization of testing 
software. 

4. 2 These approaches would interface with spacecraft/experiment standard hardware 
and provide a minimum overall GSE inventory. 


reproducibility op tee 
ORIGINAL PAGE IS POOR 


$. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Qual.Assur 


.□ 


Engineering □ Testing I I 

Manufacturing! I Materiel! 1 

6. DELTA RISK (increase, No Change, Decrease) 

Performance Decrease Schedule Decrease 

7. RATIONALE FOR RISK ASSESSMENT: 


Reliability All areas [HI 


.□ 


Cost Decrease 


7.1 The standardization of testing GSE and software will result in standard test 
methods and ability to compare and share test results among programs; col- 
lection of test data on a common basis will decrease performance risk on 
individual programs. 


7. 2 Use of multi- application or standard-design GSE will allow utilization of pre- 
tested or proven equipment, thereby decreasing schedule uncertainties and in- 
creasing probability of delivering duplicate equipment to firm cost targets. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
1. LCP No. : 3-1 2. CATEGORY : DOCUME NTATION 

3. LOW COST PRACTICE ; Use Low-Cost-Driving CDRL's 

3.1 Stress a completely finished — not to be exceeded' UDKL in each Phase L/D 
RFP. 

(a) Identify each document based on need/worth/cost analysis. 

(b) Limit CDRL to minimums in terms of quantity, page count, number of 
updates, number of copies, etc. 

(c) Establish target ceiling cost for total CDRL documentation. 


U. RESULT OP IMPLEMENTATION : 

ll.l Avoid the practice of having contractors propose excessive CDRL in proposals in 
response to RFP which is frequently non-specific re: CDRL. 

4. 2 Use of target cost will e raj±iasize minimum documentation costs to meet realistic 
minimum documentation needs in future programs. 


5. AREA. AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering □ Testing □ Reliability Q All areas lu I 

Manufacturing^] Materield Qual .Assnr, I I Q| 

6. DELTA RISK (Increase, No Change, Decrease) 

Performance No Change Schedule Decrease $ os -t Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 True spacecraft performance risks will not be impacted by eliminating excessive 
documentation typical of traditional programs. 

7. 2 Schedule and cost risks will both tend to be decreased by the elimination of un- 
necessary documentation and allowing more concentration of contractor efforts 
directly on programmatic problems. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
1. LCP No, : 3-2 2, CATEGORY: DOCUMENTATION 

3. LOW COST PRACTICE: Documentation- Need Evaluation Form 


3 t ; — — — 

Develop a documentation- need evaluation form for use by NASA in analyzing 
cost-effectiveness of each document placed on CDRL. Include such information 
as: 

(a) Use intended -- by whom, how frequently. 

(b) Content, format page count. 

(c) Specific need(s). 

(d) Copies, distribution /control intended. 

(e) Target cost of document and its maintenance. 


U. RESULT OF IMPLEMENTATION : 

[; # 1 Assure that the need and cost effectivenss of each CDRL item has been evaluated 
prior to RFP release. 

4.2 Provides a firm basis for establishing a mandatory-minimum CDRL in accordance 
with recorded and approved needs. 


5 . AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering 1 1 

Testing | | 

Reliability! 1 

All Areas f m) 

Manufacturing! 1 

Mate riel (ZD 

Qual.Assur. CU 

□ 

DELTA RISK (increase. No Change . 

, Decrease) 


Performance 

No change 

Schedule No change 

Cost No change 

RATIONALE FOR RISK ASSESSMENT: 
Performance, schedule, and 

cost risks will not be 

impacted by realistic 


tation need evaluations. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LGP No. : 3.3 2. CATEGORY : DOCUMENTATION 

3. LOW COST PRACTICE : Establish a Central Documentation Function _____ 

3.1 Establish a NASA Central (at Hdqtrs. or at Centers) Documentation Control 
Team. 

3.2 Establish minimum- documentation list (with guidelines) for use in future pro- 
grams; list to be based on cost/need/worth evaluations. 

3. 3 Monitor RFPs, proposals and programs to assure documentation requirements 
are minimum and cost-effective. 


U. RESULT OF IMPLEMENTATION : 

lj ]_ ChangsNASA historically high documentation practices based on "desirements" 
to low cost practices based on cost/need analyses. 

4. 2 Reduce costs for documentation in future programs. 


g . AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering f~~l Testing 1 1 Reliability [ 1 AH Areas |M| 

Manufacturing ! I Materiel!.,. ] Qual.Assur, □ □ 

6. DELTA RISK (increase. No Change, Decrease) 

Performance No Change Schedule No Change Cos t Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Performance and schedule risks will not be changed by this practice as the 
vital data relative to these risks will still be documented in minimum form. 

7 . 2 Cost overrun risk will tend to decrease because of the firmer estimates possible 
on an austere minimum set of documentation. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
1. LCP No, : 3-4 2. CATEGORY : DOCUMENTATION 

3. LOW COST PRACTICE : Use Direct Contact in Lieu of Paperwork 

•3*1 Establish an operating practice for maintining NASA visibility of contractor 
efforts by direct contact; use more NASA representatives in contractor plants 
in lieu of massive docunentation requirements and documentation cycles. 


lu RESULT OF IMPLEMENTATION : 

li.l Considerable paperwork and documentation update cycles which traditionally 
have been required to provide NASA visibility of contractor efforts can be 
eliminated by using more NASA reps in Contractor plants to physically 
observe contractor efforts, report interim information to NASA via telephone 
and log sheets, and provide expedited program decisions. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 
Engineering □ Testing 
Manufacturing!!!] Material I I 


Reliability | [ 

Qual.Assur. □ 


All Areas 




.□ 


6. DELTA RIS K (increase, No Change, Decrease) 

Performance No Change Schedule Decrease Cost Decrease 

7. RATIONALE FOR RISK ASSESSMENT: 


7.1 Spacecraft performance risks will not be impacted by this documenation avoidanc 
practice because the same necessary data will be provided, but in less costly 
manner. 


7. 2 Schedule and cost risks will both be reduced via this practice if NASA personnel 
are assigned responsibility and authority to make decisions with accelerated 
turn-around time and use of less documentation processing. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 
1. LCP No, ; 3-5 2. CATEGORY : DOCUMENTATIO N 

3. LOW COST PRACTICE : Use of Contractor Internal Documentation 


Page 1 of 1 


3,1 ^ Uri ^ S PhaSe B Stu f ies have contractors define the "internal documents" 
they have generated for similar programs and propose which of these docu 
ments could be used in lieu of traditional CDRL items. 

3 * 2 ST3 ^c i al* 1 NAS A* f docam . entation where possible without conversion to 

special NASA formats emphasizing use of informal working papers (oue rating 

tio 8 ns) d ) eC1S1 ° n l0gS ’ faCt SheetSj com Puter printout formats (with user instruc- 


h. RESULT OF IMPLEMENTATION : 

Ufl , redUC | docume . ntation ^ ts on a program by eliminating the traditional 

Mng 56t5 0t f ° r thC PU1P ° Se ° f managing and 


3. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering Q Testing □ Reliability Q 

Manufacturing!!] Material!!] Qual.Assur. CH 

6. DELTA RISK (increase, No Change, Decrease) 

Performance _No change Schedule No change Cos t No chance 

7. RATIONALE FOR RISK ASSESSMENT: 


AH areas 




.□ 


7.1 Performance, schedule, and cost risks will not be impacted by elimination duo 
cate data generation and transformation of data from one format to anothel 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 


LCPP-15 

Page lof I 


1. LCP No, : 3-6 2. CATEGORY ; DOCUMENTATION 

3« LOW COST PRACTICE : Lower Cost Engineering Drawings for NASA Unmanned Sp ace 

Programs 

3* 1 Implement a low-cost drawing spec which eliminates the need for and use of 

drawing forms 1 and 2 now required/allowed by MIL-D-1000 and MIL-STD-100A. 

3.2 Urge contractors to modify existing contractor drafting manuals which most 
likely are based on Form 1 and 2 requirements 


U, RESULT OF IMPLEMENTATION : 

h.l Lower drawing costs for unmanned space programs having only one to five 

spacecraft in program by eliminating excessive drawing requirements initially 
intended for aircraft programs having large production runs. 


5. AREA AFFECTED & DEGREE OF COST 

IMPACT: (High, Moderate, Low) 


Engineering 1^1 Testing j ] 

Reliability ] 

□ 

Manufacturingdl MaterielO 

Qual.Assur. [ m! 

□ 

6. DELTA RISK (increase. No Change 

, Decrease) 


Performance change 

Schedule No change Cost 

Decrease 


7, RATIONALE FOR RISK ASSESSMENT : 

7.1 Performance and schedule risks will not be impacted by eliminating excessive 
drawing preparation and drawing control requirements. 

Cost risks will be reduced by avoiding the possibility of getting Form 1 and/or 
Form 2 practices (because of contractor traditional habits and/or Drafting 
Practic Manual), and associated costs even when Form 3 is called for in RFP's. 


7.2 



LCPP-l 5 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 2 

1. LCP No. : 3-7 2. CATEGORY : DOCUMENTATION 

3. LOW COST PRACTICE: °ata Bank to Identify Documentation Costs and Cost Reduc- 

tion Areas 

3»1 Develop a documentation cost data bank to obtain better visibility of estimated 
and actual costs. 

3. 2 Distribute data for use by program offices (NASA and Contractor) in cost/worth 
tradeoffs of new documentation requirements. 

3. 3 Require contractors to estimate costs (in proposals) and provide actual costs (in 
program reports) to cover all contractor documents and related costs, including 
such categories as: 

(a) CDRL items 

(b) Contractor internal documentation 
(continued next page) 

U. RESULT OF IMPLEMENTATION : 

U.l Establish a common and credible data base for use in documentation cost/ 
worth tradeoffs in future programs. 

Allows reduced documentation costs in future programs by providing dollar 
base for elimination of unnecessary and/or marginal documents. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering □ Testing □ Reliability □ All 

Manufacturing!!] Material!!!] Qual.Assur. CH 

6. DELTA RISK (Increase, No Change, Decrease) 

Performance No change Schedule No change Cost 


Areas M 


.□ 


No change 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Risks will not be impacted by use of credible data in the performance of docu- 
mentation cost/worth tradeoff evaluations. 
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1. LCP No.: 


mjmLz RISK I MPACT OF LOW-COST PRA^th. 
-ili 2. CATEGORY : MANUFACTUR ING 


3. LOW COST PRACTICE; Increased Use of Simplified Tooling 



3. 2 Use hard/controlled tooling for control ~ • 

replacement of hardware item in the factory or fa’ aVlteW 


U. RESULT OF IMPLEMENTATION ; 

fc.l Uee of soft tooling in lieu of hard tooling will reduce tooling costs. 

Surveillance and special control of non-deliverable tooling can be reduced. 


.□ 

.□ 


5. ^ mt AFFECTED 4 DEGREE OF COST IMPACT ; (High, Moderate, low) 

Engineering □ Testing □ Reliability® " 

Manufacturing® Materiel® Qual.Assur.® 

6. DE LTA RISK (Increase, No Change, Decrease) 

Performance _No change _ Schedule No change Cost No change 
? • ^TIONALE FOR RISK ASSESSMENT : 

7,1 — 
therefore unchanged. tics, performance risk is 

7.2 Minor decreases in schedule ricV „ a. 

no-change category has been assigned. ^ - ° Verrun risk ma y occur, but a 
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LCPP-15 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE P&gg 1 0 f 1 

1 * ; — ? 2. CATEGORY : MANUFACTURING 

3. LOW COST PRACTICE : Use of Contractor Proven Manufacturing Processes 

3.1 Permit increased, use of proven manufacturing processes' ol — 

contractor or subcontractor rather than impose strict conformance 
to Government process /control specs. 


U. RESULT OF IMPLEMENTATION : 

U.l Use of existing processes, which can be validated by NASA, will eliminate 

need for preparation of special new documentation and process verification 
procedures. 


5._AR£A AFFECTED & DEGRE E OF COST IMPACT : (High, Moderate, Low) 
Engineering Q Testing Q] Reliability [~| 

Materiel ^ Qual.Assur. [ m] 


Manufacturing 


M 


6. DELTA RISK (increase. No Change, Decrease) 

Performance No Change Schedule No Stonge Qost No Change 

7. RATIONALE FOR RISK ASSESSMENT: 


.□ 

.□ 


7.1 


On basis that NASA can review and validate the existing process as adequate 
there is no change in impact upon performance, schedule, or cost risk. 
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Lcpp-is; 


1. LCP No.: 


DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page i of 2 
Jill 2. CATEGORY : PROGRAM DEFINITION AND CHANGES 


3. LOW COST PRACTICE : Emphasize Earlier and More Complete Program Definition 

3 Intensify NASA and Contractor Phase A/B efforts to achieve during Phase B, 
the scope, level and degree of program definition traditionally achieved after 
the Phase C/D preliminary design review. 


.3. 2 Require inclusion of NASA program definition /planning lists as part of the 
Phase B RFP. Require lists to be as complete as possible and in the same 
format as they will appear in the Phase C/D RFP. Include lists of: 

(a) design requirements, (b) performance requirements, (c) new technology 
requirements, (d) technical specs, (e) management and control specs, (f) major 
hardware /software items (including quantities), (g) lists of government 
facilities and GSE to be available for the program, (h) Data Requirements List. 

(continued on next page) 

U. RESULT OF IMPLEMENTATION: 


U.l Will provide much more complete program definition prior to the preparation 
of Phase C/D RFP and will allow NASA to commit to a much more firmly 
fixed program in the RFP. 

4. 2 Can avoid much of the cost growth (approximately 97%) experienced during 
Phase C/D of some historical programs by eliminating program definition 
oversights and subsequent "changes" to correct such oversight. 


5. _AREA AFFECTED & DECREE OF COST IMPACT : (High, Moderate,! Low) 
Engineering □ Testing □ 

Manufacturing! I Materield] 


Reliability! ] 
Qual.Assur. □ 


All Areas 


.0 

.□ 


6. DELTA RISK (Increase, No Change, Decrease) 

Performance Decrease Schedule Decrease G ost Decrease 

7. RATIONALE FOR RISK ASSESSMENT: 


Performance, schedule, and cost risks can all be decreased by oostulating 
a complete and near realistic strawman program, definition early in Phase B, 
firming up the final program definition via tradeoff analysis of all interacting 
definition variables prior to the end of Phase B. 

Costs and cost risks will also tend to be decreased by allowing more of the 
"true program" to be costed in Phase B and rechecked/refined during the 
short Phase C/D proposal response time. 


A-26 


LOCKHEED MISSILES & SPACE COMPANY 



LCPP-15 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE 


Page 2 of 2 


f 


LCP No: 5-1 2. CATEGORY PROGRAM DEFINITION AND CHANGES 

LOW COST PRACTICE: Emphasize Earlier and More Complete Program Definition 

3. 3 Refine and complete the definitionfclanning lists (as a result of tradeoff studies) 
during Phase B. 

3.4 Encourage selected system, subsystem, and equipm ent designs to be extended 
beyond the normal conceptual design level during Phase B when they are neces- 
sary to complete the degree of program definition desired. 


LCPP-15' 

D_ATA SHEET - RISK I MPACT OF LOW-COST PRACTICE Page 1 of 1 

1# — N ' ° - ; — 2. CATEGORY : PROGRAM DEFINITION AND CHAN GES 

**• COST PRACTICE: Program Definition Assessment Check List 

3.1 Estab lish and use a -Program Definition Assessm ent Check List fpnArr' , 

inSe planning tycl e and^Ts^ elements early 

Include penetrating questions in the PDACL concerning the statu * rW™ f 
completeness and degree of firmness of parh Pn • me sta ^ us » degree of 
Phase C/D RFP. Stress need S? 100<? „ i * Planning list for use in the 
Phase B. ° % com P lete Program definition during 

If any PD list cannot be completed 100% prior to the end of Pv, aoo r 
written justifications of why it can't be how and ^en It w m be ' eqU1 ‘' e 

k. RESULT OF IMPLEMENTATION: 


3. 2 


3. 3 


U.l 


4. 2 


TT . lists 

gree ot definition between programs. versus cie- 


S._*Rja AF FECTED & DEGREE OF COST IMPACT , (H igh; Moderate, Low) 


Engineering Q Testing | | 

Reliability | | 

All Areas |h I 

Manufacturing!!!] Materiel 1 1 

Qual.Assur. CZ3 

□ 

6. DELTA RISK (Increase. Nn 

, Decrease) 


Performance No Change 

Schedule No change 

Cost No Chari crp 

- — g z. 

7. RATIONALE FOR RISK ASSESSMENT: 


7,1 ?l h ; S Pra J tiC ° wil ] not impact program risks directly because the check list 
a verification device and is not a requirements document. 

2 impaCtS , due \° better Program definition and individual Low Cost Prac 

tices proposed to date are reported on separate data sheets. 
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1. LGP No.: 


LCPP-15 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
5 _ 3 2 CATEGORY: PROGRAM DEFINITION AND CHANGES 


3. LOW COST PRACTICE: Establish Minimum. Change Policy During Phase C/D 

3.1 


l 


After thorough and complete program definition in Phase B, establish a minimum 
change policy for Phase C/D. 


3.2 Strive for no changes. 

3. 3 Reject all cost increasing changes that are not mandatory to meet minimum pro 
gram objectives /requirements. 

3.4 Require thorough analysis of the impact of each change on the total program be- 
fore approving a change (including the possibility of triggering more changes 
and total costs for same). 


it. RESULT OF IMPLEMENTATION : 

Avoid a program cost growth due to desirable but non-mandatory changes. 

4.2 Reduces overall program costs. 


5 . AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering 
Manufacturing' 


H 


M 


Testing 

Materiel 


M 


Reliability [l"1 

rs 

Qual.Assur. 


.□ 


.D 


6. DELTA RISK (Increase, No Change, Decrease) 

Performance No change Schedule Decrease Cos t Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Performce risk will not be altered because changes required to meet mandatory- 
minimum performance requirements will still be accomplished. 

7.2 Schedule and cost overrun risks will be decreased by eliminating many "improve' 
ment only" type changes and the tendency of these changes to "snowball" into a 
series of schedule delays or cost increases 
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LCPP-15; 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
1« ; LCP No. : 5-4 2. CATEGORY : PROGRAM DEFINITION AND CHANGES 

LOW COS T PRACTICE ; Provide Incentives for Minimizing Cost- Incre ase Changes 

Provide dollar incentives for both NASA and contractors to eliminate cost 
increasing changes: 

(a) Increase contractor fee for all NASA-directed changes which increase 
program costs. 

(b) Decrease or omit fee on all contractor-proposed cost-increasing changes. 

(c) Provide cost awards for NASA and contractor personnel who suggest signi- 
ficant cost-decreasing changes during Phase C/D. 


U. RESULT OF IMPLEMENTATION : 

ii.l Will focus attention on reducing changes and associated costs. 

4.2 Will tend to eliminate "modest improvement" changes and cascading of desirable 
but unnecessary minor changes. 

4. 3 Will tend to reduce the large program cost growth experienced in Phase C/D 
of historical programs. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

nr 


Engineering |_H| Testing 


M 


Manufacturing® Materiel® Qual.Assur* 

6. DELTA RISK (increase, No Change, Decrease) 

Performance No change Schedule Decrease 


.□ 


.□ 


Cost 


Decrease 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Performance risks will not be altered as a result of eliminating changes 

which are not absolutely necessary to meet minimum performance require - 
, ments. 

7. 2 Schedule and cost overrun risks may be indirectly decreased by reducing the 
overall number of changes, the possibility for cascades of changes and reduc- 
ing the unplanned, time consuming rework forced by changes. 
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LCPP-15J 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 
1 , LGP ; No., : 5-5 2, CATEGORY : PROGRAM DEFINITION AND CHAN GES 

3. LOW COST PRACTICE: Low-Cost Program Practice Check List 


3*1 Prepare and distribute a check list of potential low cost program practices for 
use by NASA Program managers /planners. Provide backup data explaining 
the result of implementation and the rationale of risk impact (could be a set of 
data sheets similar to these LCP Risk Impact Data Sheets). 

3. 2 Require that NASA Program Office and contractor fill out check list designating 
which practices are being incorporated in a program. 

3.3 Require written justification for using a traditional (high-cost) practice when 
an alternative low cost practice is available. 

3.4 Request NASA and contractors to identify additional low cost alternative prac- 
tices, and evaluate cost and risk impacts, to expand the initial check list during 
Phase B studies. 

U. RESULT OF IMPLEMENTATION : 

U.l Will focus attention on low cost program practices and get wide-spread contribu- 
tion toward their indentification and use. 


$, AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering 1 1 Testing I 1 

Reliability | 1 

All areas | h| 

Manufacturing!!!] Materiel!!!] 

Qual.Assur. {HI 

□ 

DELTA RISK (Increase, No Change 

, Decrease) 


Performance No Change 

Schedule N ° Change 

Cost No Change 


7. RATIONALE FOR RISK ASSESSMENT : 

7,1 This practice will not impact risks directly. Separate risk assessments 

will have been made or will be required for each individual low-cost practice 
incorporated. 
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LCPP-15 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LGP No. : 6-1 2. CATEGORY ; PROGRAM MANAGEMENT (Organiz ation) 

3. LOW COST PRACTICE : Low-Cost Oriented Program Staff 

3.1 Continue to encourage 1 assignment of Program Managers, both lN-ASA and 
Contractor, who have strong "low cost" working philosophies as well as 
technical /management capability. 

3.2 Require assignment to Program Staff of a Design-to-Cost ? Life Cycle Cost, 
or Value Engineering Specialist(s) to pursue program cost reduction (and 
risk assessment) and minimal target cost objectives throughout all p hases 
of the program, particularly during the early phases before designs are 
frozen. 

3. 3 Establish a general minimum-cost policy that all program participants must 
challenge the need for and evaluate carefully any cost-increase elements. 


U. RESULT OF IMPLEMENTATION ; 

Provide a continuing effort emphasing minimum cost and cost- reduction. 


4.2 Increase emphasis on lower-cost at the working level. 

4. 3 Applies attention to the "people problem" in directing attention to non- traditional 
aspects of lower-cost programs. 


5. AREA AFFECTED & DEGREE OF COST IMPACT ; (High, Moderate, Low) 


Engineering □ Testing □ 
Manufacturing! I Material I 1 


Reliability | | All Areas H 




Qual.Assur 

6. DELTA RISK (increase. No Change, Decrease) 

Performance No Change Schedule No Change Cost Decrease 


□ 


7. RATIONALE FOR RISK ASSESSMENT; 


7.1 Most of the low-cost methodology adapted will have no impact on increased 

performance or schedule risk. Each cost reduction approach will be thoroughly 
analyzed for impact before implementation. 

7. 2 The general emphasis placed upon minimum cost will be a strong driver not 
only to reduce cost but to reduce the probability of program cost overrun. 
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LCPP-15 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page__l of 1 

1. LCP No, : 6-2 2. CATEGORY: PROGRAM MANAGEMENT (Cost Records / 

3. LOW COST PRACTICE : New Set of Low Cost CER's (Cost Estimating Relaionsiiips) ' ^ 

3.1 Develop a set of low-cost Cost Estimating Relationships (CERs) to be used, 
in lieu of historical CERs for obtaining very austere target costs for pro- 
grams. 

3.2 Provide new CERs to contractors for reference and use as abase for developing 
additional low-cost approaches. 

3. 3 As new cost data accumulates, the low-cost CERs can be "validated" and updated 
and eventually replace the current and historical "high cost" CERs. 


k, RESULT OF IMPLEMENTATION : 

U.l Setting a low target cost will place pressure upon the program to execute at 
a lower cost than the traditional programs. 

4.2 The "average" actual costs from programs using the low-cost CER's will be 
significantly lower than historical programs although some minor overruns 
of the low-cost targets may occur. 


6 


AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

All Areas 

DELTA RISK (Increase, No Change, Decrease) 

Performance No Change Schedule No Change Q ost Increas 


Engineering J_j Testing □ 


Manufacturing! I 


Materiel 


□ 


Reliability [ j 
Qual.Assur, CU 


0 

□ 


e 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 The use of low-cost CER's has no impact upon hardware performance or 
program schedule. 

7.2 The establishment of austere low-cost targets upon programs may in some 
cases result in a slightly increased possibility of overrunning the target cost. 
However, the thorough pre-planning and careful costing required to stay under 
the initial target, and holding back a contingency fund, will reduce the overrun 
probability. 
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■l ay 


DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 


1. LGP No.: 


2. CATEGORY : PROGRAM MANAGE MENT (Interfaces/ 

Meetings) 


3* LOW COST PRACTICE: Small Program Teams and Minimum Interfacing 


3.1 Assign a minimum- size iNAbA team to manage program and interface wi CT 
contractor(s) ; reviewing and/or approving contractor program, or contribu- 
ting analyses, testing, etc. 

3. 2 If possible, representative(s) of NASA team should be on-site with contractor 
office to expedite interface communications and reduce paperwork by 
substituting personal contact. 

3. 3 Limit formal technical reviews to system and subsystem level; hold informal 

reviews at component or black box level. Designate minimum- quantity of NASA 
and contractor personnel at review meetings. 


U. RESULT OF IMPLEMENTATION : 


U.l Reduction in quantity of interface meetings and reduction of specialist review- 
ing/approving design. 

4, 2 With reduced formal reviews, contractor documentation and personnel to sup- 
port the reviews can be reduced. 


$. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering [hJ Testing □ 

Manufacturing!!!] Materiel! 1 


Reliability |^| 
Qual.Assur. !ZH 


.□ 


.□ 


6, DELTA i RISK (increase. No Change, Decrease) 

Performance No Ghange Schedule No Change c os t No Ch ange 

7. RATIONALE FOR RISK ASSESSMENT : j 

7.1 The reduction in customer /contractor interfacing personnel should improve 

the quality of review and focus attention onto primary areas important to pro- 
gram success and cost. 

7. 2 Providing for less formal and expedited review of data will tend to improve 
schedule accomplishment. 
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LCPP-l 5 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LCP No, : 6 ~ 4 2. CATEGORY : PROGRAM MANAGEMENT (Cost Re cords/ 

" Estimating) 

3. LOW COST PRACTICE: Visibility of Program Practice Cost Impact (Cost Records/ 

Estimates) ' 

— - ■ 

Require a revised contractor cost reporting system which will provide 
separate labor cost summaries for functional areas such as Systems En- 
gineering, Reliability, QA, etc. , against any hardware level down through 
the component level. 

3. 2 Provide also for separate cost totaling and tracking of impact of selected 

cost-driver practices in areas such as Documentation, Testing, Configuration/ 
Data Management, Program Planning /Control. 


U. RESULT OF IMPLEMENTATION : 

U.l Identification of costs on a program at various hardwave levels contributed by the 
various functional organizations, thereby allowing post-analysis of impact of 
practices such as depth of QA surveillance, engineering time spent on docum- 
entation, cost of reliability testing, and similar. 

U.2 Cost visibility offered will aid in alteration of methods to reduce cost, 
focusing on the main cost drivers. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High,, Moderate, Low) 


Engineering 1 1 

Testing | | 

Reliability | ] All Areas 

_0 

Manufacturing! 1 

Materiel f~H 

Qual.Assur. 1 1 

□ 


6. DELTA RIS K (increase. No Change, Decrease) 

Performance No Change Schedule No Chan S e G ost Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Requiring only a change in cost recording, this practice does not have any 
impact on hardware performance nor on program schedule. 

7. 2 Identifying the cost contribution of various program practices at various hard 
ware levels will increase the program cost tracking capability and improve 
cost control and will tend to decrease cost overrun risk. 
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1. LGP No.: 


DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page_J_ofJ_ 

2 CATEGORY: PROGRAM MANAGEMENT (Planning/ 

Control) 


3. LOW COST PRACTICE : Standardized Cost/Schedule Contro l Systems 

3.1 Standarize cost/schedule reporting systems for contractor compliance; 
use same system for all NASA Centers. 

3.2 Make cost/schedule data requirements compatible with data needed by con- 
tractor for internal program management; same data to be used by NASA and 
contractor. 

3. 3 Eliminate use of 533 forms and need for hand-copying of data summaries 
thereon. 

3.4 Coordinate with DoUto establish single standard system for DoD/NASA. 


U. RESULT OF IMPLEMENTATION : 

More complete data available to NASA (copies of computer printouts) in lieu 
of very general 533 data, without cost increase to contractors. 

4.2 Standardized contractor reporting systems will reduce overall contractor costs 
(primarily overhead costs). 

4. 3 Improved correlation of data within NASA on different programs (part of new 
cost data bank). 


5, AREA AFFECTED: & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering □ Testing □ Reliability Q Program Mgt_[M] 

Manuf acturingCZI Materiell_J , Qual^Assur, CZD _i — 1 

6. DELTA RISK (increase. No Change, Decrease) 

Performance No Change Schedule No Change Gos t No Change 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 The change in cost/schedule recording and reporting has no influence on peifor- 
mance risk. 

7. 2 Although there may be some improved visibility to NASA of cost data, the impact 
on schedule or cost overrun risk is essentially unchanged. 
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LCPP-15 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1* LG^ N o . : _._6_- 6 2. CATEGORY : PROGRAM MANAGEM ENT (Cost 

3. LOW COST PRACTICE ; _ Standard WBS and D.g,‘ for C?rt gfLonts 

3.1 Establish specific and standard NASA-wide WBS deflations for specific 

functions and hardware with appropriate breakdowns under each. 

3. 2 Pre-assigned WBS sub-breakdown with WBS dictionary descriptions are re- 
commended. ^ 


U. RESULT- OF IMPLEMENTATION : 

k* 1 Actionary 6 ^ C ° St ° f contractor Preparation of program-peculiar WBS and 

4 * 2 ^ ll L Pr r ide a c . ommon ' cost - collection system with no question about what cost 
elements were included or omitted under each titled item of the breakdown. 

4. 3 Will allow comparison of costs program-to-program and identification of special 
cost-driver elements. p u 1 


5.-.A R5A AFFECTED & DEGREE OF COST IMPACT ; (High, Moderate, Low) 
Engineering □ Testing | | 

Manufacturing!!!] Materiel! i 


Reliability [ [ Program Mgt |m| 

Qual.Assur. □ □ 


^ • DELTA RIS K (increase. No Change, Decrease) 

Performance _ No Change Schedule No Change Cost Decrease 
7. RATIONALE FOR risk ASSESSMENT ; 

7.1 There is no impact of this practice upon performance and schedule risk. 

7. 2 The standard cataloging of all program cost elements will provide a better 
base for collection of cost data, use of which can improve confidence in esti- 
mates of later programs, thereby tending to decrease cost overrun risks. 
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LCPP "*t 5] 

DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 

1. LCP No. : 6-7 2. CATEGORY: PROGRAM MANGEMENT 

JC5 st Records /Estimating) 

3. LOW COST PRACTICE ; Common Cost Data Bank (Cost Records) 


■ Establish a NASA-wide cost data bank which vail allow collection/recording 

of costs down through major component level and with breakouts for direct 
labor.material, overhead, for both NASA and contractors. 

3.2 The bank should record initial estimated costs as well as actual program 
costs and should contain listings of "normal" or "alternate low-cost" 
approaches which were on the actual program. 


U. RESULT OF IMPLEMENTATION : 

^•1 Provide data for cost impact analyses of normal versus low-cost approaches 
and preparation of low-cost CER's for use on future programs. 

v 

4. 2 Provide basis for establishment of firm and lower-cost targets for new pro- 
gram Design-to-Cost. 

4. 3 Allow easier evaluation of contractor cost proposals and easier analysis and 
monitoring of principal program cost drivers. 


MJ 

□ 


5. AREA AFFECTED & DEGREE OF GOST IMPACT : (High, Moderate, Low) 

Engineering □ Testing | 1 Reliability | | All Areas 

Manufacturing!!!] MaterielLJ Qual.Assur. CD 

6. DELTA RISK (Increase, No Change, Decrease) 

Performance No Change Schedule No Change Q os t Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

■7 *V . ■ • ‘ 

* This practice deals principally with cost data; therefore program performance 
and schedule risk are not impacted. 

7. 2 Use of the data bank will provide a firm base for estimating of new and follow- 
programs and designation of program ground rules impacting cost. This will 
have a significant effect upon reducing probability of program cost overruns. 
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LCPP-1 5' 


DATA SHE ET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1 * LCP No. ; 2. CATEGORY : PROGRAM MANAGEMENT (Interfa ces/ 

3. IOW COST PRACTICE! Experimenter Interface Improvement Meeting s) 

3.1 Establish an early and iirm technical interface between the Expenmenterls) 
and spacecraft management teams to assure compatible interfaces with 
minimum downstream change potential. 

3. 2 Try for completion of experiment development (coordinated with spacecraft) 
before start of Phase C/D on spacecraft. 

3. 3 Require spacecraft contractors review /signoff of experiment interface drawings 


U. RESULT OF IMPLEMENTATION ; 

U.l Early interfacing will allow "compromises" on both experiment(s) and space, 
craft in arriving at optimized low-cost configurations and hardware. 

4. 2 Prior proving of experiment in development will solidify spacecraft require- 
ments, thereby leaving minimum program change potential. 


$._AREA AF FECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 

Engineering 0 Testing 0 Reliability □ Exp erimenters |M| 
Manufacturing!^ MaterielD Qual.Assur. □ 

6. DELTA RISK (increase. No Change, Decrease) 

Performance Decrease Schedule Decrease Qost Decrease 

7. RATIONALE FOR RISK ASSESSMENT : 

7.1 The performance risk for the combined experiment/spacecraft will be reduced 

by early compatibility analyses and coordination of functional interfaces, test- 
ing, etc. ’ 


7.2 


Early and continued interfacing will have a strong influence on reducing possible 
changes to experiment and/or spacecraft, thereby decreasing schedule and 
cost risk. 
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i. LCP No. : 2. CATEGORY : PROGRAM MAN AGEMENT (Config. Mgt) 

3. LOW COST PRACTICE : Reduce Program Hardw are Traceability Requirements 

Reduce or eliminate requirements for limited" quantity "(less than live iliglTt 
articles), production programs in areas of part and material traceability, 
identification, material certifications, numbering and codes, etc.) and configura- 
tion traceability. 


U. RESULT OF IMPLEMENTATION : 

^•1 Hardware traceability and associated documentation and costs, intended pri 
marily for large r-quantity, field- replaceable hardware, will be reduced. 


5. AREA AFFECTED & DEGREE OF COST 

IMPACT : (High , Moderate . Low ) 


Engineering H Testing Q 

Reliability [h | 

n 

Manufacturing M Materiel® 

Qual.Assur, (m 1 

□ 

6, DELTA RISK (increase. No Chanee 

, Decrease ) 


Performance No change 

Schedule Increase Gost 

Increase 

7. RATIONALE FOR RISK ASSESSMENT: 


•1 The importance of maintaining detail records of specific configuration, usually 
down through the parts level, on programs with very few flight articles is 
que stionnable regarding useful statistical information. All hardware units 
are usually built about the same time and are probably functionally inter- 
changeable. Impact on hardware performance is negligible. 

7. 2 Because there is a possible small advantage in being able to trace more 

quickly the cause of ground of flight failure and incorporate changes on re- 
maining hardware, there may be minor potential increased risk to program 
schedule and cost; the degree of risk should be assessed on each new program. 
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1. LGP No. : 7-1 2. CATEGORY i QUALITY ASSURANCE 

3. LOW COST PRACTICE: Rewrite of QA Specification 


J.l Prepare and issue a low-cost revised version of the QA Requirements Spec: 

NHB 5300.4 (IB) or portion of NHB 5300.4 (ID). 

3.2 A reduction of QA surveillance is proposed for the rewrite; however, inspec- 
tion requirements would remain essentially unchanged. 

3. 3 Emphasize the need to apply the low-cost approaches at all levels of procurement; 
prime contractor, subcontractor, and suppliers. 


U. RESULT OF IMPLEMENTATION : 

U.l Will clarify the specific functions to be performed by QA as opposed to 
functions to be performed by other contractor organizations to provide a 
high-quality product. 

4.2 Will reduce cost of QA surveillance on a program. 

4. 3 Will impact subcontract or supplier costs also (a large percentage of total 
program costs are frequently allocated there). 


g. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering □ 

Testing I I 

Reliability [HI 

□ 

ManufacturingCZl 

Materiel 1 1 

Qual.Assur. CHI 

□ 


6, DELTA RISK (increase, No Change, Decrease) 

Performance No Change Schedule No Chan § e Cos t N ° Change 

7. RATIONALE FOR RISK ASSESSMENT : 

Proposed reduction of specific QA functions are not intended to alter the 
basic inspection, nor the testing surveillance performed on the product 
which would remain essentially unchanged. 

7. 2 Although the cost of the Q A effort will be reduced, the probability of program 
cost overrun will also be unchanged. 
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2. CATEGORY : QUALITY ASSURANCE 


3 * — C0ST PaACTICS ; Use of Contractor Quality Manual - Low-Cost Version 


3.1 


3.2 


Request contractor to prepare and submit a "low-cost version of his Quality 
Manual which features low-cost practices listed here and elsewhere and in- 
cludes a basic program Quality Plan. 

Validate the new Manual and permit use in subsequent contractor proposals in 
lieu of submitting a newprogram-peculiar Quality Plan. Require that program- 
peculiar changes or deviations be submitted as an addendum to the Low Cost 
Quality Manual. 


k, RESULT OF IMPLEMENTATION : 

U.l Eliminates need to submit a new Quality Plan with each proposal (large degree 
of redundancy among various plans). 

4. 2 Use of a low-cost Quality Manual would be a strong forcing function toward re- 
duction of program costs. 

4. 3 Use of validated contractor Quality Manuals would accelerate implementation of 
low-cost practices. 


$._ARgA_A FFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 
Engineering □ Testing □ Reliability □ All Other 
Manufacturing!!] Materiel!] Qual.Assur. E 
6. DELTA, RISK (increase, No Change, Decrease) 

Performance No Change Schedule No Chancre 


□ 


7. RATIONALE FOR RISK ASSESSMENT: 


Cost No Change 


7.1 


Many of the Quality Assurance functions can be reduced without degrading 
the product quality or performance. Also, each of the elements of a new° 
low-cost Quality Manual would be reviewed carefully by NASA before valida- 
tion, lowering the probability that any unforeseen program risk would be 
incorporated. Therefore, no change in program risk will result 
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1. LGP No. : 7-3 2. CATEGORY : QUALITY ASSURANCE 

3. LOW GOST PRACTICE : Lower-Cost Material R e view 

Allow use of working-level personnel (contractor only) in lieu of formal MRB 
for material review and disposition of all but program- critical hardware. 
Provide for review of log records by Customer QA rep. 

3. 2 Delegate total customer "Contracting Officer Approval" to the Governemnt 
QA Rep (participating in the MRB) to approve all "use as is" or "rework" 
decision. 

3. 3 Limit the preparation of non- conforming material reports. 


U. RESULT OF IMPLEMENTATION : 

li.l Less paperwork and fewer delays in processing non- conforming hardware. 
4.2 Allows concentrating attention of MRB on critical elements of the program. 


$. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering M 

Testing Q 

Reliability Jm] 

n 

Manufacturing!!!! Materiel!!!] i 

Qual.Assur. (3 

□ 

DELTA RISK (increase. No Change, 

Decrease ) 


Performance 

No Change, 

Schedule Decrease Cost 

Decrease 

RATIONALE FOR RISK ASSESSMENT : 




7*1 The informal activity in lieu of formal MRB on non-critical hardware items 
will not involve increased performance risk because the dispositions are re- 
viewed by the Customer QA Rep and cannot pass unnoticed. Also, non- conform- 
ing material reports are "after the fact" documentation and have no influence 
On corrective action relating to the non- conforming hardware and will therefore 
not impact performance risk. 

7. 2 The reduction of delays in processing hardware will tend to decrease schedule and 
cost overrun risks. 
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1. LCP No, : 7-4 2. CATEGORY ; QUALITY ASSURANCE f 

3. LOW COST PRACTICE : Reduce QA Effort on P reviously- Qualified Hardware 

3.1 Where previously- qualified and/or flight proven hardware is used, require - 

reduced QA support for surveillance of these items (considerably less than require^ 
for newly-developed or first usage equipment). 

' $ 

3. 2 Apply also to standard hardware where usage experience has demonstrated 

product quality. I 



U. RESULT OF IMPLEMENTATION: 



U.l General, simplification of QA surveillance effort, fewer detail inspections 
required, and general reduction of QA costs. 

4. 2 With increased use of NASA standard hardware, implementation could result 
in lower costs at an early date. 


$. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 
Engineering □ Testing Q Reliability 


.□ 


Manufacturing! 


:□ 


Materiel 


□ 


Qual.Assur. Hi 


.□ 


6. DELTA RISK (Increase, No Change, Decrease) 

Performance No Change Schedule No Change c os t 


No Change 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Where adequate qualification and/or satisfactory flight experience has been 
demonstrated on a hardware item, nothing is gained from further detailed 
inspection. No change in program risk therefore is forecast for applying 
the minimum QA surveillance to the "proven" hardware elements. 
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7-5 2. CATEGORY: QUALITY ASSURANCE 


3. LOW COST PRACTICE : Reduce Quantity of Inspection Points 

3.1 Empahsize that inspection of hardware be performed at the highest, possible 

assembly level to reduce the redun of inspections. Inspect at total assem- 

bly if all parts are still visible. Require minimum inspection points in the 
manufacturing process, emphasizing product quality in the completed artic 

3. 2 equire that critical dimensions requiring inspection be identified on the Enginee 
ing drawing; all other dimensions need not be verified by Inspection. 


li. RESULT OF IMPLEMENTATION : 

Reduction in the ’.inspection cost of parts and subassemblies when their princl 
pal' characteristics can be verified in later assemblies or installations. 

4. 2 Emphasis on critical dimensions only will focus inspection efforts on those 
’ • areas most important to product quality and intended use of hardware. 

4.3 Showing inspection requirements on Engineering drawing will assure early 

Engineering /Inspection coordination and possibly eliminate need for separate 
Inspection Instructions. 


3. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering CD Testing I 1 

Reliability 1 1 

□ 

Manufacturing! 1 MaterielLD 

Qual.Assur, CD 

□ 

6. DELTA RISK (Increase, No Change 

, Decrease) 


Performance No Change 

Schedule No Change Q 0 st 

No Change 

7. RATIONALE FOR RISK ASSESSMENT: 


7 t Inspection is often performed on all characteristics of hardware at all levels 
of assembly, frequently where the elements are not altered during the manu- 
facturing process. Elimination of many of these essentially redundant inspec- 
tion points and the concentration on critical characteristics only will not alter 
the program risk. 
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X. UjQQ. 8 2. CATEGORY : SPECIFICATIONS 

3. LOW..,. COST PRACTICE: _ Revision or Rewrite of Current Specs to Low-Cost Versi ons 

3.1 Revise existing high- cost driving specs 'to include o' nly mandatory and lowe r- 
cost requirements. Typical examples: 

(a) MIL-D-1000 Eng. Drawings Reqts. (c) NHB 5300. k(lB) Quality Assurance 

(b) MIL-STD-499 System Eng. Reqts. (d) MIL-STD-883 Testing Requirements 


3. 2 Establish an altered format, for new or revised specs which includes the 
rationale" explaining the reasons for the requirements, the cost-driving 
characteristics, the low-cost objectives, and the altered program risk if 


U. RESULT OF IMPLEMENTATION : 

U.l Specifications drive a significant portion of program costs. Selection of the 
principal cost-driver specs and issuance of low-cost versions of same will 
have an immediate effect on reduction of iiew-program costs. 

4. 2 Inclusion of the "rationale" in the new/revised specs will aid in their quick 

adoption and implementation of lower- cost methods in NASA and throughout 
industry. ° 


5.. AR EA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 
Engineering []] Testing | ] Reliability [ [ Ail areas 

Manufacturing!!] MaterielC] Qual.Assur. E] 

6. DELTA RISK (increase, No Change, Decrease) 

Performance No Change Schedule No Change Q os t No Change 

7. RATIONALE FOR RISK ASSESSMENT; 


□ 


7.1 


7. 2 


Each low-cost specification before issuance will be preceded by a risk assess- 
ment; the increase or decrease in program risk will be explained in the "rational 
portion of the spec. The altered risk, if any, will therefore be known to the 
cognizant program manager before the spec is applied to a specific program. 

Most of the proposed low-cost changes to specs listed above in 3.1 can be imple- 
mented with literally no alternation^or a decrease of program risk. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LCP No, : 9-2 2. CATEGORY : SPECIFICATIONS 

3. LOW COST PRACTICE : Cost-Effectiveness Analysis of Spec Applications 

3.1 Require a cost effectiveness analysis to be performed on application of each 
spec to a program before designating it as a compliance document in RFP. 

3. 2 Invite contractors to propose deviations to specs or recommend alternate 
specs either in pre-RFP coordination or in the RFP. Request cost- saving 
estimates for alternate spec recommendations. 

3. 3 Also, require preliminary cost-tradeoff analysis of new program-peculiar 
performance specs prior to issuance of the spec. 


U. RESULT OF IMPLEMENTATION : 

U.l Early considerations of the impact of specs on program cost will tend to reduce 
the types and quantity of specs listed for compliance in the RFP and thereby 
provide decrease in average contractor cost proposal response. 

4. 2 Early involvement of contractors in spec "analysis" will prvide' additional cost- 
saving approaches. 

4. 3 Early performance /cost tradeoffs will focus attention on cost-driver areas of 
the performance requirements and provide initial potential cost reductions. 


3. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, 
Engineering □ Testing I 1 Reliability ! | 

ManufacturingCZI Materieldl Qual.Assur, HZU 

6. DELTA RISK (increase. No Change, Decrease) 

Performance Decrease Schedule No Change 


Low) 

All Areas 


0 

□ 


Cost Decrease 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 A preliminary assessment of cost impact for performance specs will probably re 
suit in lowering of the performance requirements; this will, on the average, 
make the performance level more attainable, thereby lowering the risk. 


7.2 The focusing of attention on cost-driving characteristics of specs will result 
in better NASA/ contractor cost estimates and a potential decrease in cost 
overrun risk. 
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1. LGP No. : 9-3 2. CATEGORY : SPECIFICATIONS 

3. LOW COST PRACTICE : Do Not Apply General Spec s to Contractors 

3.1 Re-emphasize elimination or reduction of the application of general specs 
for Reliability, Safety, Maintainability, QA, Systems Engineering, Human 
Factors, and similar to a program unless specific paragraphs of these 
specs are identified (and clarified where necessary to add specific ness 
for compliance in the RFP/contract, 

3. 2 Require that each segment of each general spec be reviewed prior to inclusion 
in the RFP to assure that the requirements are mandatory for the specific 
program. 


U. RESULT OF IMPLEMENTATION : 

ii.l Full application of these general specs results frequently in over- responsive- 
ness by contractors in applying large numbers of support personnel to a pro- 
gram. Tailoring of the general spec to the specific needs of a program and 
specifying portions of spec and degree of compliance required will guide 
contractors to lower-cost responses. 

4. 2 Reduction in ambiguity of what and how much effort is needed in specific 
support areas. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering [m] Testing [m] 
Manufacturing! I Mate rielGZl 


Reliability [hJ 
Qual.Assur. 


.□ 


.□ 


6. DELTA RISK (increase. No Change, Decrease) 

Schedule No Change Cost Decrease 


Performance No Change 


7. RATIONALE FOR RISK ASSESSMENT : 

7.1 Clarification of type and degree of spec compliance will allow better cost 
estimates by NASA/ contractor; tendency for cost overruns will be reduced 
moderately. 

7. 2 There will usually be no impact upon performance and schedule risk because 

requirements are being made clearer, not altered. In cases where requirements 
are reduced, NASA will make the risk assessment at that time. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1, LGP No. : 9-4 2. CATEGORY : SPECIFICATIONS 

3 LOW COST PRACTICE : Establish Central Spec Control and. Coordination 

’3.1 Establish a central NASA coordinating activity for generation, control, 
and release of specs for total NASA, 

3.2 Emphasize standardization and cost-effectiveness of specs and rewrite of 
high-cost driver specs to low-cost versions. 

3. 3 Review newly-proposed specs and spec lists for new programs to assure 
they are minimum and cost-effective. 


U. RESULT OF IMPLEMENTATION : 

U.l Reduce the quantity of different specs applied to contractors. 

4.2 Allow contractors to establish low-cost operating procedures to comply with 
the new cost effective specs. 


5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering 1 1 

Manufacturing! 1 

Testing 1 1 

Materiel 11 

Reliability | | 

Qual.Assur. CU 

All Areas 

FI 

n 

6. DELTA RISK (Increase, No Change 

, Decrease ) 



Performance 

No Change 

Schedule No Change 

Cost No Change 

7. RATIONALE FOR RISK ASSESSMENT: 


7,1 The mandatory elements of specs which impact hardware performance risk 
or schedule and cost risk would be retained in the new specs. No change in 
program risk would result. 
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1. LCP No. : 9-5 2, CATEGORY : SPECIFICATIONS 

3. LOW COST PRACTICE : Use of a Reduced "Standard" Spec List 

3.1 Reduce to a minimum level the quantity and types of NASA and DoD specs 
in the spec "library" (used as a general reference in selecting specs for 
inclusion in RFP's as compliance documents). 

3.2 Establish a "standard" set of specs and spec list for future NASA programs. 

3. 3 Reduce or eliminate rewrites of existing specs by NASA Centers into program- 
peculiar specs where- minor changes only are required in the "standard"; state 
the modification in the RFP/SOW/Contract. 


li. RESULT OF IMPLEMENTATION : 

li.l Reduce the average quantity of specs applied to a particular program. 

4.2 Eliminate the non-mandatory specs, requirements of which have tended to 
support some non-mandatory work at contractors. 


$ . AREA AFFECTED & DEGREE OF GOST IMPACT : (High, Moderate, Low) 


Engineering 1 1 

Testing I I 

Reliability! [ 

Ail Areas 

|m| 

Manufacturing! 1 

Materiel! 1 

Qual.Assur, d3 


n 


6, DELTA RISK (increase, No Change, Decrease) 

Performance No Ghan S e Schedule No Cos t N ° 

7. RATIONALE FOR RISK ASSESSMENT : 

As part of the spec analysis preceding removal of any spec from the "library", 
the potential impact on typical program risk will be assessed. In general, 
those mandatory specs which involve risk will be retained in the spec library. 
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i. LGP No. : 10-1 2. CATEGORY : TESTING 

3. LOW COST PRACTICE: Optimize Testing for Minimum Cost 


Require tradeoff on each program of which tests shall be performed at 
component, subsystem, or spacecraft level to obtain minimum overall 
test cost. 

3.2 Require combination where possible of system qualification and acceptance 
testing using a single proto-flight test article. 

3. 3 Emphasize high cost of system testing; limit to overall workmanship and 

end-to-end functional testing. More thorough testing if required should be 
performed at subassembly, component, and subsystem level. (This approach 
is made easier by modularizing equipment packaging). 

3.4 Emphasize minimum retesting for Standard Components or Standard Subsystem 
Modules. 

4. Result of Implementation 

U,1 System- tests involve large facilities and large crews of personnel. All 
possible effort to reduce time of system test and reduce probability of 
failures occurring (by prior cost-effective testing at lower assembly levels) 
will result in significant savings. 

4.2 Combining qual and acceptance tests will save cost of a test article, but 
also provide more cost-effective use of test personnel and facilities. 


3 . AREA AFFECTED & DEGREE OF COST IMPACT : (High* Moderate, Low) 


Engineering □ Testing ("h] 

Manufacturing! 1 Materiel ! 1 


Reliability! j 

,D 


Qual.Assur, 

6. DELTA RISK (increase. No Change, Decrease) 

Performance No Change Schedule Decrease 


Cost 


□ 

□ 


Decrease 


7. RATIONALE FOR RISK ASSESSMENT : 

7«1 A well-planned integrated test program will result from this practice; the 
limiting of system test to end-to-end functional testing probably will have 
little impact on flight performance risk if rigorous testing has been done 
at the subassembly and component level. 


7.2 The simplified system test, combined with upstream component/ sub system 
testing will tend to decrease the probability of schedule slippage or cost 
overrun risk. 
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DATA SHEET - RISK IMPACT OF LOW-COST PRACTICE Page 1 of 1 

1. LGP No, : 10-2 2. CATEGORY : TESTING 

3, LOW COST PRACTICE : Test Software and Equipment Standardization 

3.1 Emphasize standardizing testing equipment and test instrumentation to rhaxi- 
mum possible on a single program and among similar programs (minimum 
quantity of different test equipment modules). 

3. 2 Require use of commercially- available rather than new program-peculiar 
test equipment where feasible. 

3. 3 Require use of the same test software and test equipment (or portions thereof) 
at component, subsystem, and spacecraft level where possible. 


k, RESULT OF IMPLEMENTATION : 

U.l Multiple usage of same piece of test equipment reduces cost. 

4. 2 Use of standard equipment allows standardization of spacecraft hardware 

interfaces. 

4. 3 Use of same test method/test equipment at various levels of assembly increases 
total test data available tor statistical reliability analysis of specific spacecraft 
hardware elements and increases confidence level in test results. 

4.4 Use of proven commerical test equipment lowers development costs. 

5. AREA AFFECTED & DEGREE OF COST IMPACT : (High, Moderate, Low) 


Engineering ["""l Testing |"h] 

Reliability! 1 

n 

ManufacturingEZI MaterielLj 

Qual.Assur. CH 

n 

DELTA RISK (Increase, No Change 

, Decrease) 


Performance Decrease 

Schedule Decrease Cost 

Decrease 

RATIONALE FOR RISK ASSESSMENT: 


7.1 Use of same test equipment and software on multiple units of the same hard- 
ware elements increases the confidence level of the test resvilts; provides 
more data points for same piece of hardware. This will tend to reduce the 
hardware performance risk. 

7.2 Use of standardized test equipment, proven on a previous program will 
eliminate some "first usage" problems and tend to decrease the schedule and 
cost overrun risks due to testing. 


) 
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1.0 General Background : MIL-STD-810B, Environmental Test Methods superseded 
the original' A 'version of the specification in June 1967. Since that time numerous 
additions have been made to the document, the most recent being issued as a series 
of new pages to be incorporated, date September 20/1970. Originally intended as 
a standard series of methods for the environmental testing of aircraft, the docum- 
ent has undergone amendments to transform it into a standard suitable for space- 
craft use. The results of such a process will be discussed in this EM as part of • 
the text. MIL-STD-883A, Test Methods & Procedures for Microelectronics; date Novem- 
ber 15/I97li overlaps MIL-STD-810B, because • it specifies the environments to be 
considered when applying test to determine the merit of micro -electronic devices, 
including Integrated circuits (ICs). This document is highly detailed, current, 
and technically explicit in the methods and conditions it imposes. The MIL-STD 
does not cover hybridized circuitry, but is in wide use throughout the micro- 
electronics industry. It will.be discussed. extensively in the ensuing text. 

1.1. Applicability of the Documents : In the broadest sense, MIL-STD-810B sets forth' 

standards for environmental testing at all hardware levels from piece-parts to com- 
plete vehicles. Many of the requirements do not apply to spacecraft, either manned, 
or unmanned, and the requirements which do apply tend to be dated, and require 
revision. MIL-STD- 883 A has an impressive sub-tier of other specifications and 
standards associated with it, many of which would appear to be un-necessary. The 
subtier will be treated in this EM, in terms of its applicability and pertinence 
to not only the MIL-STD, but also to unmanned space programs generally. 

1.3. Cost Aspects of the MIL-STDS : EM LCPP-3 in this series of Engineering Memo- 

randa showed the influence of both MIL-STDs upon two programs; MVM-73 and the Atmos- 
phere Explorer. Both of these programs undertook an elaborate series of environment- 
al tests on items of hardware from "black-box" up through the assembled flight-ready 
vehicle; and both programs employed electronic devices containing a heavy complement 
of micro-electronic elements. From programmatic cost data available from these prog- 
rams the inference was made that program costs were driven from 3-h% TPC by MIL- 
STD-8103, and from 7. 9-8.0$ by MIL-STD- 883 . (No 'A 1 version had been issued at the 
time the programs were undertaken). It was inferred also that program savings of 
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1.5-2. 5$ of program costs might be saved by mo'dification of the requirements 
of both • MIL-STDs, and- a combination of. them into one overall document where 
possible . 

1.1;. Analysis Objective ; The major objective of this analysis of the two MIL- 
Stds was to examine the validity of this cost-savings hypothesis, and confirm, 
or deny its validity. 

2.0. Detailed Analysis MIL-STD-810B : This MIL-STD is one of the better examples 
of a specification which "Grew like Topsy". Originally issued in June 1961; as 

an 'A' version, the'B' version was issued in June 1967. Prior to the 'A' version 
released in 1961;-, the first version MIL-STD-810 was first invoked in 1952, 
during the Korean war. The standard was totall aircraft oriented in its applic- 
ation, and served its purpose for a limited time. As high performance supersonic, 
all-weather, all climates pursuit aircraft became widely used by the US armed 
services in the middle, and late 1950's; the environmental standard no longer 
sufficed. Additional criteria wer added, until in 1 961; a formal re-issue took 
place. Realizing the the environmental test methods and criteria were not suit- 
able, not applicable, or both, for spacecraft, yet another issue was made in 
June 1967, which included chapters on Space Simulation, and Acoustical vibrat- 
ion. Again, the issue was made by the US armed services, and again the issue 
did not suffice for long. As better understanding of environmental impingement 
on space hardware became current throughout the aerospace industry, a'B' version 
was issued in October 1969. During the period of US involvement in the Viet Nam 
war, new rapid-fire, multi -barrelled armament was added to military aircraft. 
Other new armament included large caliber fast-fire weaponry (fire rate: fast 
fire is any delivery rate up to 1500 rounds per minute; vs rapid-fire which is 
any delivery rate from 1500-7000 and above rounds per minute). These novel weap- 
ons imparted severe vibration to aircraft structures, and such vibrations were 
suspected to contribute materially to structural fatigue. Thus in September 1970 
amendments were issued for incorporation into MIL STD 810B, which dealt with 
methods of simulating the affects of armament vibration on not only structures, 
but also sensitive equipment mounted to such structures. Minor changes continued 
to be issued on this subject, the last being October 20 1970. The STD, as of 
Feb 1975, contains all these amendments. 

2.1. Applicability of the STD : As has been so often the case, NASA was "force- 
fitted" with MIL-STD-810, 810A, 810B, and all amendments issued to date. Most of 
the material, included in the STD does not apply to spacecraft, as stated prev- 
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iously, so for purposes of reference the STD is compiled as follows: 

Sections 1 thru $ furnish data concerning test methods, referenced docum- 
ents, test conditions, particulars of test facilities, test sequences, records, 
and other information pertinent to the overall conduct of tests in compliance 
with the standard. Methods are given, commencing with #$ 00 , and continuing 

thru #$12. Methods #$13 thru $19 are furnished but these have a subscript 
numeric .1 to signify that the content of these methods sections have been 
amended from their original state. The numbers and titles are: 


$00 Altitude 

$01 High Temperature _ 

$02 Low Temperature 

$03 Temperature Shock 

$Gl* Temperature - Altitude 

$0$ Sunshine 

$06 Rain 

$07 Humidity 

$08 Fungus 

$09 Salt Fog - 

$10 Dust Test . 

$11 Explosive Atmosphere 


Aircraft Oriented. 

Aircraft Equipment Oriented 

Aircraft Equipment Oriented 

Aircraft Equipment Oriented 

Aircraft Electronic Equipment 

Equipment Earth's Atmosphere 

Equipment Exposure to Rain 

Equipment in Humid Atmosphere 

Equipment in Fungus promoting 
Atmospheric Environment . 

Equip' t in Dust/Sand Envr'mt 
Equip 't operation in Explosive 


$13.1 Acceleration 
$14.1 Vibration 
* $l$.l Acoustical Noise 
$16.1 Shock 

. $17.1 Space Simulation 


Aircraft/Air craft Equip Oriented 
Aircraft, Air- Air Missiles 
Aircraft, Missiles, Spacecraft 
Aircraft, Missiles, Spacecraft 
Spacecraft, Spacecraft Equip 'mt 


$18.1 Temperature - Humidity - Altitude Aircraft, Aircraft Equip' t.. 
$19.1 Gunfire Vibration, Armament. Aircraft, Aircraft Equip' t. 

In addition to all of the above, there is method $12 which deals with leakage. 
The two methods specified are water, and hot oil immersion. The methods seek 
to determine whether or not a sealed container leaks when submerged in fluid, 
and the integrity of hermet seals, gaskets, etc. The test method was excluded 
from the major listing above, as at present it is under scrutiny, and may be 
deleted from the STD, with another leakage test method being included in its 
place. 








” 
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As may be seen from the above listing, only three test methods apply to space- 
vehicles "directly, yet it is common practice for NASA RFPs to invoke the STD 
routinely, without mention of the specific method sections of pertinence. 

These three methods are responsible for much of the costs associated with 
environmental testing of spacecraft, ; and spacecraft flight hardware; and of 
the three, the largest cost contributor is method 517.1 Space Simulation . 

2.1.1. Specifics of $17.1 Space Simulation : This test method is described as 
intended for the evaluation of space vehicle component's, space vehicle subsys- 
tems, and complete space vehicles including installed equipment. The method 
sets forth in considerable detail seven (7) procedures by which such an eval- 
uation, or set of evaluations, shall be made. The 7 procedures are as follows: 

I. Radiant Heat Transfer effect of space, simulated solar spectrum. 

II. Cold Absorbing Nature of space, infrared heat source. 

III. Cold Surfaces of chamber as source of calculated temperature impression 
upon test object. 

IV. Orbital skin temperature, by infrared heat source. 

V. Heat Transfer simulation for components, by use of a conductive, 
temperature controlled surface . . 

VI. Simulation of the, pressure effects of space, by low pressure chamber 

VII. Heat transfer by use of a cryogenic heat sink plus vehicle heaters. 

To qualify the terms of these procedures and furnish essential test information, 
chamber pressures are given for the 7 methods, tabulations of solar energy in 
space,' radiation characteristic of the planets in the solar system, solar sim- 
ulator classification tabulations, a tabulation of solar' electro-magnetic dis- 
tribution of energy, details of a heat flue simulator, a- radiant heat sink, 
details of test equipment set up, and operating procedures for conducting the 
tests. Definition of the terms and nomenclature associated with space simulation 
tests, and thermal- vac cum chambers used for these tests is also provided. 

As such, the method is well documented, well written, well ordered as to subject 
matter, and highly precise in terms of the requirements spectra iset forth. 

2.1.2. Costs of the Method : The method of space simulation described was first 
released in 196 9 (Oct 20). At that time little was known of the long term affects 
of a space environment upon sophisticated space hardware, especially at the 
complete vehicle level. Also at that time, no expense was spared to bring to 

a close the successful series of lunar missions. Costs were considered secondary 
to the success of a mission, especially in the Apollo series involving as they did 
human life . 
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To give some indication of the costs of space 'simulation testing the following 1 
data are -of ‘value ; 

1. Costs to procure a space simulation facility similar to Chamber "A" 
at JSEC. 

o Chamber structure & installation of 60' dia x 65' high unit. $13. 5m 

o Vehicle preparation and assy building.... $ 1.75m 

o Chamber support equipment (tank farm for gases/consumables) .$ 0.75m 

. o Automatic Check-out Equipment (per set ) .$12. 00m 

Total $28. 00m 

The dollar totals are in 1975 dollars, today the costs would be almost double. 

2. Operating costs per hour, full scale tests, large vehicle... $1500.00 

3. Costs to install LMSO facility approx l/2 size of JSFC chamber $11. 00m 

1*. Cost to operate $850 per hour. 

5. Costs to operate a small chamber such as chamber G at JSFC $275 »00/hr . 
Chamber is approx 7' dia by 10* ht, initial cost $3.7Um. (1965 $s) 

As a general rule, full scale vehicle tests are run on a 21+ hour per day per 
unit chamber. This requires 3 labor shifts of approx 70 persons per shift, ' 
and the hourly rate of expenditure is $1500 per hour -as stated. These figures 
are merely those costs of running the test which for a test such as the service 
module and the LEM ran for 30 days, at a cost of over 1 million. Test planning 
test preparation, and site activation cost at least 25 million more when the 
test teams at the contractor's facility and the test personnel at NASA are 
considered. LMSC does not suggest that these dollar figures are excessive in 
a 20+ billion dollar program, especially when the outcomes of the tests cond- 
ucted 'afforded the confidence to proceed with the manned" program. The sums in- 
volved are Very costly indeed for a program not requiring such sophistication, 
is the point at issue. 

2.1.3. Space Simulation Testing,; Needed or Not ? Currently there is a considerable 
controversy surrdunding whether or not space simulation testing is any longer 
necessary, particularly in the light of the NASA trend toward the use of stand- 
ardized flight qualified space hardware. This controversy involves the level of 
hardware at which such testing is to do undertaken; the determinimg factors 
being the relative costs at the component and the higher-order levels. One 
school of thought contends that it makes little sense to test at the complete 
vehicle level when vehicles involved are relatively small, and unmanned. This 
appears to be the dominant thinking when cost reduction is considered vital 
to the success of NASA programs. 
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Another school of thought, diametrically opposed to the first, contends that 
the costs of full scale vehicle testing are money well spent, if concealed 
defects are revealed by the process, especially where the vehicle involved 
is "one only to be built". The prevailing opinion, would appear to be the first,, 
with the generally expressed opinion: "Space Simulation testing is too costly 
to conduct at the vehicle level, do it at the component level, and only do it 
for those components which are new and not flight qualified" being heard most 
frequently. 

2.ll3.1. RFP Specifics : LMSC recommends that the RFP be used as the vehicle i 

for resolving the question of level at which to undertake this costly form of 
testing. Not only must the MIL-STD-810B be invoked for details of the method, 
but the level must be specified, and any exclusions to the 7 procedures for 
test set forth in the MIL-STD must be stated. Of those 7 procedures, some are 
either/or choices; and in these cases, the one required should be identified. 

If this practice is followed, considerable costs can be avoided. Contractors 


who have incurred the expense of installing large scale space simulation test 
facilities, tend to oe proponents of their use, for the obvious reason of cost 
recovery, and installation expense amortization. Similarly, NASA personnel who 
have been involved with the -design, procurement, operation, and maintenance of 
such facilities tend to favor perpetuation of such facilities. To some degree, 

NASA facilities for space simulation are necessary, as contractors not possess- 
ing such facilities can, and do make use of the NASA test complexes. There is 
no simple answer to whether or not space simulation testing should be used; 
but it would appear obvious that the subject should be carefully considered 
in the' light of the cost/ worth aspects of such testing prior to its implement- 
ation. Above all, specify in the RFP exactly what is believed necessary, and 
invoke precise section and methods of the MIL-STD rather than invoking it as 
a "blanket:"'; 'program standard. 

2.1.4. Potential Cost Savings Summary : To illustrate the cost savings possible 
if space simulation testing is either reduced to the corapoent level or eliminated 
the following tabulated estimates: are presented for two programs MVM-73 & A.E. 

Total Test Costs $TPC . Total Env Test Costs Total Space Sim. ^ 


MVM-73 

■A.E. 


18.0$ TPC 
16.2$ TPC 


3.8$TPC 

U.o$tpc 


1.7$ TPC 
1.9$ TPC 


2 . 1 $ 

2 . 1 $ 
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As can be seen from the estimated savings tabulated, on a program costing 
approx 100 million such as the MVM-73,' a potential savings of 2 million is an 
appreciable amount. Recognition of this fact has caused NASA to modify require- | 
ments for both the LST program, and the JFL program SeaSat. (JPL is considered, 
as though it were a NASA agency for purposes of this study). On these programs 
space simulation testing has been required at the component level only, and 
MSFC and JPL environmental test specifications have been invoked rather than 

i : - 

the general MIL-ST D-810.B . 

2.2. General Recommenda.tion; LMSG recommends that NASA no longer invoke MIL— STD- 
810B except as a reference/guide .. NASA should ^standardize from among the exist- 
ing environmental test methods/ specifications extant at MSFC, JSC, and GSFC, 
an environmental test methods specification suitable for the forthcoming, stand- 
ard components era. This Test Standard for NASA use, should include Space Sim- 
ulation testing at the component level only, and only that, when components to 
.be used are of novel, not flight-proved designs. In cases where space simulation 
appears essential at the vehicle level, detailed cost trade studies should be 
made, before undertaking the vehicle level tests. Such studies will confirm or 
deny the cost effectiveness 'of tests at the vehicle level. 

When MIL-STD-BlOB Is no longer invoked, or invoked only as a reference and 
guide,; other specifications/standards called out by the standard are also negated 

as the dependency sub-tier. These are: 

o MU -S -9 01. Shock Tests, High Impact, Shipboard Machinery, Equipment & 

' Systems, Requirements for. 

o MIL-E-2036. Enclosure for Electric, and Electronic Equipment, Naval 
Shipboard. 

o MIL-G-5572. Gasoline, Aviation Grades 80/87, 100/130, 115/1^5- 
o MIL-C-9U35. Chamber, Explosion-proof Testing. 

0 MIL-C-U5662. Calibration of Standards. ( also called in NHB 33Q0.1j(lB)) 
As may be seen, almost all of the above documents do not apply to NASA operational 
hardware in any manner. 

3.0 Detailed Analysis MIL-ST D- 883 A : This MIL-STD- establishes in-depth uniform 
methods and procedures for testing microelectronic devices, including basic 
envir omental te sts to determine resistance to deleterious effects of natural 
elements and conditions surrounding military and space operations, and physical 
and electrical tests. By definition contained within the STD, the term device // 
includes such items as monolithic, multi-chip, film, and hybrid microcircuits. 
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microcircuit arrays, and the elements from which the arrays are formed. 

The standard has three purposes; 

1. The specification of suitable conditions obtainable in the laborat- 
ory, which give test results equivalent to actual service conditions: 
existing in the field, and the assurance of reproducibility of the 

results of the tests* ( 

2. To describe all of the test methods of a similar character in current 
use by the joint services and NASA which appear in the various 
specifications issued by those agencies, in one standard. The object 
of such combination is the uniformity of test methods and the conserv- 
ation of test equipment and facilities, as well as manhours. 

' 3. To provide a series of test methods applicable to devices not covered 
by approved Military/NASA specifications, standards, and/or drawings. 

As is usual with MIL -STDs a documents sub-tier of dependencies is included, 

•and these documents may be listed as follows; 


Specifications ; 


Fe de ral: QQS-5 71 


Solder, Tin alloy; tin lead alloy; 


. . ' and lead alloy. 

MILITARY - * 

. MH-F-1U256 - Flux spidering. Liquid (Rosin base) 

MIL-S-I 950 OE - Semi conductor devices; general specification for 
MIL-M- 385 IQ - Microcircuits; general specification for 
' ' ‘ MIL C U£662 - Calibration System Requirements. 

STANDARDS ; j ! , 

MIL- STD-280- Definition of Terms for Equipment Divisions 
KIL-STD-781 - Reliability Tests, Exponential .Distribution 
MIL-STD-I 313 - Microelectronics Terms & Definitions. 

In total, all of these documents combined give the impressive figure of £$0 
printed pages of material which must be read, understood, and applied. In 
addition to all of the above verbiage contained in these documents, NASA imposes 
yet another requirement, primarily upon the manufacturers of the microelectronic 
devices. This document is NHB 5300.It(3C) and is called Line Certification 
Requirements for Microcircuits. In essence, the document informs the manufact- 
urer that if he wishes to sell his product to NASA, he must process it in the 
manner ‘prescribed, and inspect it per the criteria set forth in the publication- 
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As a document, this segment of the NHB series is highly detailed, of sound 
technical content, and covers every major facet of the manufacturing processes j 
in current use by the semiconductor/microcircuit industry. NASA was obliged to 
produce such a document (issued in May 1971) since the yield of devices from 
the production process was low, the process was expensive and non-uniform, 
supplier to supplier, and the incidence of hidden defectives within the j process 
output was high. The Handbook segment is divided into k major chapters, and while 
it is not invoked or referenced by MIL-STD- 883 A the following detail is furnished 
to provide a digest of its contents and pertinence to microcircuit production 
and acceptances _ 

Chapter 1 . Introduction . Deals with Scope, Applicability, Required Document- 
ation for Line Certification, Applicable Documents 
invoked, and Definitions of Microcircuitry Terms. 

Chapter 2 . General Requirements for Line Certification : Cites Pre-requisit- 
es for line certification, Phases of Line Certification, Loss 
of Line Certification’ (criteria for), Re-instatement of Line Cert- 
ification, Groups of Circuits to be Manufactured on Certified Line, 
Levels of Control, Test Options, Fabrication Documents, Location 
of Process Steps, Environmental Control of Test Areas, Handling 
of Sub-strates, Water Control, Purity of Materials (criteria for) 
Corrective Action, and Facility Usage. 

Chapter 3 . Detail Requirements for Line Certification of Monolithic Micro - 
- * 

Circuits : This chapter deals with: Substrate Characterization, 
Surface Passivation, Patterning, Junction Forming, Metallization, 
v Metallized Substrate Acceptance (Level II), Scribing and Dicing, 

Die Sort Inspection, Die Mounting, Interconnect Bonding, Pre- 
Seal Visual Inspection, and Sealing. 

Chapter 1*. Guidelines for Maintenance of Line Certification for Monolithic 
Microcircuits : This chapter is a series of what purport to be 
guidelines, but since almost all of them commence with the phrase 
"The contractor shall", in effect, .they are mandates. Covered 
are the requisites for maintaining certifications of the processes 
of: Substrate Characterization, Surface Passivation, Patterning, 
Junction Formation, Metallization, Metallized Substrate Accept- 
ance (Level II), Scribing and Dicing, Die Sort Inspection, 

Die Mounting, Interconnect Bonding, Pre-Seal Visual Inspection, 
and Sealing. 
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The jHandbook segment includes also a Documents Requirements List (ERL). 

Review of- the document, and knowledge .of the subject matter jas a whole permits 
LMSC to state that the requirements are rigorous, the checks and balances are 
elaborate, and technically complex; the documentation of each of the processes 
is detailed and exhaustive, and the processes themselves represent the latest 
state-of-the-art. As such the Handbook segment is largely responsible for the 
highly reliable microcircuits routinely deliverable to NASA by the industry. 

The multiple checks and inspections both of process viability and merit, and 
quality of the end item, are essential to produce the quality level required. 
However, the entire process of obtaining, and maintaining, line certification 
is highly costly, and these costs are reflected in the unit cost of the end- 
item. Whether or not the costs are justified in the light of the possible: risk 
of major mortality occurring in NASA programs, is a moot point. To date, so far 
as is known no study has been made of the comparative risk associated with each 
production step on a basis of "Include a specific control", "Exclude a specific 
control" NASA has been of the opinion that the costs, no matter how great, were 
money well spent. In the climate of modern austerity which besets the aero- 
space industry, LMSC believes that such a study would be of benefit, and recom- 
mends that one be undertaken. LMSC conducted a cost comparison for micro- 
circuits, commercial versions vs NASA/Military requirements which show the 
costs of the specialized units to be 15-20 times that of the commercial units; 
and it is this type of result (furnished to NASA in a PER Handbook LMSC-D387518, 
30 -May* 197U) which gives rise to the LMSC recommendation. 

3.I. Analysis of Dependency Documents ; As listed in paragraph 3.0, there is a 
documentation sub-tier associated with the MIL-STD. The first of these documents 
stipulates the details of solders intended for use with semi-conductors and 
microcircuits. MIL-F-lli256 similarly, furnishes details of rosin based liquid 
fluxes recommended for use with these solders. To some degree both the Federal 
document, and the MIL specification overlap in their intent, as the solder STD 
covers solders with rosin flux cores. Both these documents are clear, concise, 
and technically satisfactory, in that they set forth the requirements for 
metallurgical configurations, chemical constituents, tests to be performed to 
verify these criteria have been met, and quality assurance and inspection prov- 
isions. 


Page 11 of 25 

LCPP-1S 


The next document in the dependency listing is MIL-S-19500E, Semi-conductor 
Devices, -General Specification for, At first glance the' reason fdr inclusion 
of this document is hot immediately apparent. Further investigation reveals 
that since MIL-STD- 883 A purports to cover hybrid circuits as well as micro- 
circuits, the MIL-SPEC is essential to cover characteristics 1 and requirements 
of the transistor type of semi-conductor. MIL-S-19500 is yet another specific- 
ation whose growth has been random. The specification was 1 first released in 
1957. Versions A,B, & C followed in the period 1958-1963, with the D version ; 
issued in 1961j. At that point there was a lull in the amendment process, until 
the E version plus Supplement IF was issued in August 1972. Amendment h followed 
in March 1973» and also at that time Appendices A & B were revised to cover 
Definitions , and Abbreviations and Symbology. In effect, what transpired was 
that as the technology of discrete semi-conductor devices was developed and 
refined, so the MIL-Spec was amended/revised to encompass the rapidly changing 
technology. The trail through the several amendments,* supplements, and append- 
ices can. only be described as tortuous. The original specification which deals 
not only with transistors, but also with diodes hovers all the technical para- 
meters, and some of the detailed requirements for manufacture to military re- 
quirements. The ubject matter thus ranges from color coding standards for diodes, 
to internation-agreements concerning standardization. The international agree- 
ments would appeal to consist of notifying all participants in the agreements 
when any proposed revision of any kind would violate the agreements aforemenioned 
Elaborate and detailed -Quality Assurance and Inspection criteria are included 
in Appendix C, which covers among other matters the criteria for Lot Tolerance 
Percentage Defective (LTPD) and a sampling table for given sampling plans. Also 
featured are Life Test, Screening and Burn-in, and Qualification procedures. 
Throughout the entire publication, there is a requirement for numerous and var- 
ied reports and documents to be submitted to the military} each evidencing Some 
portion of the overall compliance with the criteria of the specification. In 
addition to all of the foregoing content, there is the Supplement IF. This 
document of 22 pages lists a difference alpha-numeric "slash" designator for 
individual device types, and generic families of related device types. Examples 
are MIL-S-19500/1A, a transistor 2N220, to MIL-S-19500A87 (USAF) covering 
transistors 2N5838, 2N5839, 2N58i*0, and Jan-TX types of the same genus. Inter- 
spersed in this listing are diodes MIL-S-19500/l2ljE covers the diodic family 
TXLN28ii6B through TX1N2896RB, inclusive of their non TX originals. 
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Not to belabor the point "ad nauseam", this listing is an attempt to document 
the detail specification sheet associated with each type, or family of types. 

The designator Jan signifies a device which has met the specified requirements 
of the Government procurement agency, and the TX designator very loosely defin-r 
ed, signifies High-Reliability criteria have been met. To the semi-conductor 
specialist, all of this proliferation of documentation is familiar, but to the 
average design engineer, it is bewildering in the extreme. As can be seen from 
the issue date of the latest version, the document is only two years old. How- 
ever;, to make it a useful tool, it needs re-issue as a completely new document, 
merely to permit progression-through- its. requirements, without any changes to 
the basic requirements themselves. The electronics designer, contemplating in- 
clusion of a hybridized circuit in his component design, must not only search 
all the data sheets for the transistor type he wishes to use as a discrete 
element of the hybrid circuit, but also he must search MIL-M-38310A for a suit- 
able microcircuit to associate with the discrete transistor, PNPN Triode, Thy- 
ristor, op SCR (Silicon Controlled Rectifier) device). 

3.1.2. MIL-M-36310A, Military Specification, Microcircuits, General Specification for; 


This specification is another example of documentation proliferation, which 
has arisen from the attempt -to keep software abreast of the development of 
mircrocircuit technology. As with the 19300 series for discrete devices, this 
specification for microcircuits contains a supplement IE which lists by device 
type, and family > the detail specification sheets for alpha-numeric "slash" 
designators IB thru 201. A further sublisting cross-references the military 
types to similar commercial types. This supplement is dated lit Feb. 1973 . 
Amendment 2 t o the specification, dated 10 July 197b * deals exclusively with 
deletions and substitutions to the basic specification in terms of metallurgy 
percentages required, and also treats; lead finishes, case outlines, flat pack 
configurations, outlines & dimensions, etc. etc. 

3. 1.2.1. M1L-M-38310A Main 'Specification : The main document, issued 3 July 1972, 
states that the specification establishes the general requirements for mono- 
lithic, multi-chip, and hybrid microcircuits , and -the quality and reliability 
assurance requirements which must be met in procurement of microcircuits. 

This specification has its own sub-tier of documents invoked which includes: 

MIL SPECS : MIL-M-35363 Microcircuits, packaging of 

MIL-C-I 43662 Calibration System Requirements. 

MEL STDS: MIL-3TD-100A Engineering Drawing Practices. 
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MIL-STD-129 Marking' for Shipment & Storage 
- - MIL-STD-883 Test Methods & Procedures for Micro- 

electronics. * 

MIL-STD-1313 Microelectronic Terms & Definitions # 

. MIL-STD-1331 Parameters to be Controlled for the 
Specification of Microcircuits * 

Publications : 

FEDERAL : Cataloging Handbook Hh-1 Federal Supply Code for Manuf’rs 

MILITARY: NAVSHIPS 0967-190-Ii 010 . Mfrs Designating Symbols. 

NASA: NHB 5300.1*(3C) Line Certification for Microcircuits.* 

Those documents segregated with an asterisk * are 1 cross referenced in the MIL- 
STD-883 under discussion, and of particular interest is the NHB 5300*ij(3C) 
which has been discussed earlier in this review. 

• In reading the document, the user is given information on the entire manufact- 
uring process for microcircuits, and subjects such as device qualification, 
line certification, design documentation, die topography, device marking, 
lead metallurgy, bonding, arid P.A. requirements. In this latter subject. 

Product Assurance is specified as to levels and intent in a complete Appendix. 
This Appendix A is titled Product Assurance program, and it cites an elaborate 
program of checks and balances for all phases of device production. In addition 
to the program. Appendix B is furnished which details Statistical Sampling, 

Test & Inspection Procedures, including the Lot Tolerance Percentage Defective 
Method, and Hypergeometric Sampling plans for small lot sizes of 200 units, or 
lass. 'A further Appendix C is given, which gives case outline drawings with 
dimensions for T.O. cases and flat-packs, and finally, there is a Supplement 1 
which cross references devices by type designators, to commercial types of sim- 
ilar characteristics. Lastly, and invoked by Appendix B there are two military 
publications, both of which also are invoked by MIL-STD-883A. These are: 
MIL-STD-280. Definitions of Item-Levels, Item Interchangeability, 
Models, and Belated Items. . 

MIL-STD-781B Reliability Tests, Exponential Distribution. 

Respectively, these publications deal with standard terms & definitions for use 
with microcircuit devices, definitions of models both experimental, and prod- 
uction, and a technical treatise on reliability testing. This treatise, MIL- 
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STD-7813 dated 28 July 1969, outlines test levels, and test plans for rel- 
iability qualification (demonstration)', reliability production acceptance (sam- 
pling) testa, and for longevity tests. Tests are based on the Exponential, or 
Poisson Distribution, and are intended for those equipments whose mortality 
is considered to be characterised by such distributions. As might be expected, 
this STD also has a sub-tier, which will not be given here. Suffice it to say 
that there is considerable discussion as to the validity of the exponential 
distribution as a descriptor for device mortality. Experts in the field are 
challenging the distribution which has characterised electronic device mortal- 
ity since 1957 , as evidence appears to be growing, that a normal distribution, 
is a more valid. descriptor, and for long-lived items, the Weibull distribution 
is receiving wide use and attention. 

3.2 Review by Section MIL-STD-883A : Thus far, the review of MIL-STD-883A has 
been concerned with the tortuous path through the dependency documents sub- 
• tier. The manufacturer of microelectronic devices, must test his product in 
accordance not only with all of the sub-tier -requirements, but also he must 
use the methods and procedures set forth in the main text of MIL-STD-883A. 

•This publication is divided 'into five (5) sets of major tests. The categories 
for these sets of tests are:- 

0 Environmental Tests (set 1001 thru 1015.1) ........ 1 ! 

0 Mechanical Tests (set 2001.1 thru 2017) 2 

o Electrical Tests (digital) (set 3001.1 thru 3OU4 ). . • 3 
0 Electrical Tests (linear) (set 1*001 thru 1*007 )....,. I4 
: o Test Procedures (set 5001 thru 5006),. $ 

Sets 2,3, '■& 1* may be considered as a primer of test methodology and techniques 

I r~-: 

for the manufacturer of microelectronic devices. Not only are the test conditions 
cited for each test process, but also the limits are furnished for each para- 
meter of interest. In many of the areas of interest to the military, illustrat- 
ions of acceptable processing, and non acceptable conditions, have been included 
in the text. (e.g. 2010.2, 2011.1 Internal Visual, and Bond Strength, respectively) 
In the electrical Tests sets both digital and linear, the material furnished 
is illustrated by curves representative of pulse shapes, pulse rise and decay 
plots ; and required test schematic diagrams j so that performance expected can 
be visualized clearly. 
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In Test sets 3 & h which deal with electrical testing, many of the tests 
furnish data of a parametric nature, which act as inputs to equations employed 
for the calculation of device parametric merit. All such equations german to 
a given test are furnished, and the variables are defined are listed together ; 
with their specialized associated symbology. To select an example at random: 
Method 1*001, is concerned with Input offset voltage and current and 
bias current. Clause 3»U of Test Method 1*001 deals with: 

Input Offset Voltage Drift , viz: Measurement of V iQ l is made at 
Temperature T^ per section 3*1 and a second measurement at Tg 01 V^ q 2 
is amde at the second temperature. Thus r 



v io 2 - V 


T 2- T l 


It is not the intention of LMSC to review in this EM all of the test methods, 
•.their parametric implications, and expected outcomes per unit device. Such 
a practice Would be not 'only time consuming, but of little interest to per- 
sons other than device designers and manufacturers. All of the test methodol- ,, 
ogy represnts the best available state-of-the-art in -the manufacture and proof - 
of-merit of microcircuits. As may be discerned from the relatively late issue 
datejs of the major documents and their sub-tiers, to a large extent these 
publications have evolved as the necessity for the many test methods has been 
evidenced. Sets 1 and 5 of the 5 Test categories form the basis for this review, 
as it is these two sets which contribute most to the high cost of microcircuits 
used by the military and NASA, when contrasted with the cost of commercial ver- 
sions of the same basic devices. The ensuing text consideres these two categor- 
ies and their test sets in some detail. 

3.2.1 Review of Method set # 1 : This set of methods is concerned with Environ- 
mental Tests. The set includes: 

1001 Barometric Pressure, reduced (altitude operation) 

1002 Immersion 

1003 Insulation Resistance - 

100U Moisture Resistance (Current version 1001*. 1) 

1005.1 Steady-State Life 

1006 Intermittent Life 
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1007 Agree Life 

: 1 ■, 

■ - 1008.1 High Temperature Storage 

1009#1 Salt Atmosphere Corrosion 

1010.1 Temperature Cycling 

1011.1 Thermal Shock 

1012 Thermal Characteristics 

1013 Dew Point 
1011*. 1 Seal 

L... 1015 Bum-in Test. 

3. 2. 1.1. jMethod 1001 Barometric Pressure, reduced (altitude operation ) 1 

This test, routinely performed by the manufacturer at least on a small sample 
per device genus, is designed to reveal the presence/absence of voltage break- 
down failures due to the reduced dielectric strength of air and other media/ 
insulating materials at reduced pressures. Tests are made at 15, 30, 50, 70, 

100, 150, & 656K feet. 

Comment: The need for a user to repeat this test is slight, and should only 
be an influence on the users' test program 1 if the device genue is a new release. 

3. 2. 1.2. Method 1002 Immersion : This test is intended to determine the merit 
of the seal of microelectronic devices. A hot bath is used over a wide range 
of temperatures, followed by a cold bath (usually water). Transfer time hot to 
cold must not exceed 15 seconds. 

Comment : This should be a manufacturers' test only. It need not be repeated by 
users except in cases where device performance under operating conditions evid- 
ences seal deficiencies. Stock should then be tested by lot/batch on a sampling 
basis .• 

3* 2. 1.3 Method 1003 Insulation Resistance : This test is to measure the resist- 
ance offered by the insulating elements of a component/part to an impressed 
direct voltage tending to produce a leakage of current through, or on the sur- 
face of these elements. Impression voltage vaires from 10-1000 volts ± 10/. 
Comment: This is another test performed routinely, which should not be repeat- 

ed by a user unless his stock of devices evidences problems duing operation. 

3. 2. 1.1*. Method lOOtul Moisture Resistance : Performance of this test is to 
permit evaluation in an accelereated manner, the resistance of parts and con- 
stituent materials to the deteriorative effects of heat and humidity conditions 
typical of tropical environments. 
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Oossnt: This test is performed as part of the qualification prooess by the 
manufacturer.. Each f^ly of devices is so qualified, and the test is 4 ted 
o^a sampiins basis for each batch produced per genus. Users need not "peat 

space d TTd ln CaS8S ° f dimoulty - has »<> pertinence to 

apace-destined devices. Such devices are unlikely to experience the tropical ' 

ditions simulated by the test, and prior to launch the devices are afforded 

» l/ !h ° f 3 °° ntr0lled atmosphere in the Vehicle Assembly Building as 

3*2 IS M th r i nn= hr ° Und “ Pa5i ° a!i rea dy-to-launch condition. 

~ 6 h ° d IljQ5a Steady-State Life : This test is performed to establish 

a representative failure rate ft) for microelectronic devices, or to demon- 

„ ”l V T>. qUallty ° r reliability of da vices s'ubjicted to specified conditions. 

The test is adequate in all respects for the purpose intended. It is 
however, a rel iability test primarily, intended to be performed by the mfr., 

“!h r a TomTl' ratS agencies; 

■ (MAM) t T ^ ° mter (MW) Pbil -a Hate Mta service US Navy 

< ARAM), al. A user routinely refers to data furnished by such an agency 

PUrp0M 0f ohhalhirg ^vice failure rates, and should not undertake ' 
the expense of repeating the test unless there is evidence duri^^" of 

e /ice, or devices, that the life expectancy under stated conditions failure 

or he failure rate per salience period of operation, differ ml^gy 

fr^ th. Government published values, the period being shorter, or the lambda " 
being. considerably greater. 

3 ' Z ' 1 ' 3 * * 6 " — - h0<i 1006 intermittent Life; This test is similar to 1005.1 which 
proceeds it The basic difference is that the failure rate is determined ™der 
ditions designed to explore the effects of cyclic variations in electrical 

stresses between the ... and -off., state; and the attendant variations in t- 
ice . and case temperature . 

Comment : AH comments made for Mp+’nnH mn£ „ 

:: r lieth0d 1005 Preceeding, apply equally to this 

3.2.1.7 Meth od 1007 Agree Life ; This test is intended to determine failure rate 

, 6V1Ce ' ° r g6nUS ° f QSVlceSy actual system application, and operat- 

ing environment are simulated as closely as possible. 

— ’ The 'term AGREE denotes Advisory Group, for the Reliability of Electronic 
Equipment , and XndistryMlatary organisation formed in 1 95 6, to update and 
improve relaibility. This test is specifically defined in the STD, is a composite 
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Test Methods lOO^.l, and 1006. It is again a reliability test, and is required 
to conform with the requirements of MIL-STD-781 Reliability Tests, Exponential 
Distribution July 28,1969* The test should not be performed except by a device 
m nufacturer under surveillance of a Government quality/reliability investig- 
ating/certifying agency. A user should only undertake the test if government 
published data, and the test results obtained in operation for a device vary 
to a marked degree j and only then after informing a Government certifying agency 
of such an intention. (Most Government Contractors/device users have access to 
a U.S. Government Quality Assurnace Representative, who is either resident at 
the users' facilities, or can be contacted for visit/surveillance purposes in 
case of problems with a device, or genus of devices. 

3*2. 1.8. Method 1008.1 High Temperature Storage : The purpose of this test is 

to determine the effect upon devices stored at elevated temperatures for pro- 
tracted periods, with no electrical stress applied. Test conditions range from 
75°C to 1*0Q°C in 8 intermediate steps. 

Comment : This test, performed as part of the device qualification process by 

the manufacturer should not be repeated by the user, particularly when devices 
are space destined. Where storage under elevated temperature conditions is not 
contemplated, the test constitutes an unnecessary expense. 

3. 2. 1.9. Salt Atmosphere (Corrosion) Method 1009.1 : This test constitutes the 
accelerated laboratory simulation of the effects of seacoast atmospheres. 
Duration of the test period can be 21*, 1*8, 96 , or 2i*0 hours. 

Comment : Devices are protected by controlled atmosphere prior to space launch, 
either in a environmentally controlled building, a payload shroud, or both. 
Prolongued exposure to corrosive salt atmosphere thus does not apply, and 
the user need not repeat the test. 

3.2.1.10 Method 1010.1 Temperature Cycling : The purpose of this test is to 
determine the resistance of a part to exposure at extremes of high and low 
temperatures, and to the effect of alternate exposure to these extremes. It 
was originally intended to simulate the transfer of equipment to and from heated 
shelters in arctic/antarctic areas. Manufacturers ppform this test on a sampling 
basis for most device families. 

Comment ; Theuser need not repeat this test for devices whose operation will 
be in space environments, except in cases where the device(s) may be subjected 
to the variations imparted by solar heating/space frigidity. 
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3.2.1.11 Method 1011,1 Thermal Shock ; The purpose of this test is to assess 
the, resistance of the device to sudden exposure to extreme changes in temper- 
ature.. All JAN TX types arle subjected by genus to this test, on a sample/batch 
basis. 

Comment: Device users should not repeat this test, except where such extremes 

are expected to be encountered by the equipment in which the devices are usedj 
and only then when thermal shock problems become apparent in operation. 

3.2.1.12 Method 1012 Thermal Characteristics : This test is designed to deter- 
mine the case or junction temperatures (T c or T j.) heat sink temperatures (Tg), 
thermal response time, or the thermal resistance {5j _ c or 0 j _ p) oi 
microelectronic devices. 

The test is performed routinely by the device manufacturers, who publish test 1 
conditions, test criteria, and the outcomes of such tests in the device data 
sheets. Users should not repeat such tests unless there is reason to suspect 
major departures from the published thermal characteristics when devices are 
operated under normal conditions. In such cases both manufacturers, and Federal 
surveillance/Quality agencie.s should be notified of the problem, and the intent- 
ion of the user to undertake the tests. 

3.2.1.13 Method 1013 Dew Point : The test is designed to detect the presence 

of entrapped moisture within the device package, in sufficient qualtity to affect 
device parameters . 

Comment: This is a test to be performed by the device manufacturer. It should 
not be repeated by theuser, unless sharp parametric discontinuity is encountered 
with lots/batches of devices. These devices whose parameters are thus affected 
would be rejected by a user(s) and the test would be performed by physics of 
failure diagnosticians. Findings would be relayed to both Federal Quality agencies, 
and to the manufacturer concerned. 

3. 2.1. lit Method lOllt Seal : The design of the test is to determine the hermeticity 

of the seal of microelectronic devices with designed internal cavities. Essent- 
ially it is a leak rate test, performed by the manufacturer, and not intended 
to be repeated by the user. 

Comment: Users should not undertake this test excpet in cases where problems 
are encountered. 
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3.2.1.15 Method 1015.1 Burn-in Test : The purpose of this test is to screen 
outj. marginal devices, to eliminate those with -inherent defects, and those with : 
defects which may become apparent as a function of elapsed operating time. 

If the test were not . performed such devices might, in fact would be expected 
to be evidenced, as infant mortality, or early lifetime failures. The test is 
performed at maximum rated operating conditions to reveal time and stress dep-' 
endent failure modes. : 

Comment ; Considerable controversy surrounds the subject of 'screening and burn- 
in tests. Arguments for it include parameter stabilization, and the elimination 
of infant mortality items. Arguments con include the foreshortening of the life 
expectancy of the device(s), and the additional cost imposed upon the device 
by such tests. Yet further arguments rage ove'r who should perform the test, 
manufacturer, or user? In -practice, the test is performed by both parties 
frequently, and the costs are passed on to the ultimate user, the US Government. 
Since the tests are performed routinely by the manufacturer, under both user 
inspection, and government source inspection (GSI) LMSC is of the opinion that 
aerospace users, should not repeat the tests unless infant mortality problems 
arise with a given device genus. 

3.2.2. Summary oi Environmental Tests Review ; The tests prescribed in this 
section Of the STD are in general, well designed for their intended purpose; 
the determination of device "operation under a wide range of environmental con- 
ditions. In almost all cases, the tests ; prescribed are performed routinely 
by device manufacturers who must qualify their devices before offering them on 
the_ open market. Problems arise when both manufacturer and user perform the same 
tests., Such dual performance arises from the invocation of MIL-STD-883A in the 
applicable documents section of RFPs released to the aerospace industry by Govern 
ment agencies; primarily US Military and NASA. When the STD is invoked routinely, 
and with no specific direction given by such statements as this: "The contractor 
shall employ microcircuitry which complies with the standards set forth in MIL- 
STD-883A"; the user feels obliged to repeat tests to assure, himself that the 
devices he proposes to use are indeed to the standard that has been specified. 
Even specification of Jan TX types is no guarantee of the merit of the delivered 
product. ■ :! pty - 

..Further, when the devices are incorporated into hardware at the Next Highest 
Level of Assembly (NHLA), they undergo repetitions of these tests, at varying 
levels of rigor, as called out by the overlap engendered by MIL-STD-810B. This 
not only imposes hardship on the devices, but also results in additional test 
cosl. burden to a program. It is an axiom that merit cannot be injected into 
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a device by repeated testing. LMSC contends therefore that unnecessary, costly 
replication 'of testirig,is a practice to be discouraged. 

3*2.3. Review of Method Set # 5 ’• Method Set #5 is titled Test Procedures. As 
such it contains the specific procedures/techniques for control of tests 
included in sets 1 thru i* of the STD. The listing of procedures is as- follows: 
o Method 5001 Parameter Mean Value Control 
o Method 5002 Parameter Distribution Control 
< o Method 5003 Failure Analysis Procedures for Microcircuits 
o Method 500li.2 Screening Procedures 

o Method 5005.2 Qualification and Quality Conformance Procedures 
o Method 5006 Limit Testing. 

3. 2. 3.1 Method 5001 Parameter Mean Value Control : This procedure is concerned 
with the recording of Test data amassed from the conduct of any, or all tests, 
conducted as per the prescriptions of Method Sets 1 thru lj. It is not intended 
•to apply to devices whose parameters are between specified limits, but rather 
to lots of devices where the average of a given parameter, or mean value is 
controlled at some required level. In simple terms, a lot so controlled can be 
considered as a "matched set" . - , 

Comment : In order to provide the highly controlled JAN-TX designated product, 
group/batch testing for this device is necessary. In essence minima and maxima 
over a small range of extremes are set upon parametric means. From a given batch 
of devices, the yield under this form of controlled test procedure is small. 

This adds to the cost of the product, but does provide a highly uniform popul- 
ation lot-to-lot. As a routine design practice, such close control of parameters 
is to be discouraged, as unnecessarily costly. 

3. 2. 3. 2. Method 5002 Parameter Distribution Control : This procedure defines 
a technique for assuring a normal distribution for any test method listed in 
sets #3 & U of the STD. Essentially, this is a statistical technique for assuring 
that tests undertaken to assure parametric conformity with the prescribed limits 
do not vary more than ± 1 std* deviation from the parametric mean. In particular, 
not less that 12$ but not greater than 18$ of unti-s tested will fall below the 
mean - 1 sigma limit, and not greater than 18$, but not less than 12$ of units 
tested will fall above the mean + 1 sigma limit. 

Comment : This is a severly restrictive requirement, and the yield from any given 
lot or set of lots is small. The costs of such normalising of parameter distribution 

are high, and the practice should be doscouraged unless, proof is. furnished to 
the Government of its essentiality. 
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3 « 2 « 3 « 3 « Method $003 Failure Analysis Procedures for Microcircuits ; 

This procedure sets forth in detail the methods for post-mortem examination 
of failed microelectronic devices. The methodology employed is often called 
physics of failure analysis, and employs advanced analystical techniques such 
as microscopy, metallurgical analysis, chemical analysis, radiography, photo 
micrography, et al. Details of required apparatus are specified, as well as 
the particulars of all analytical tests and processes to be employed. 

Comment : While such analysis is undertaken at the facilities of device manuf- 
acturers, not all device users are equipped to undertake anayses of this level 
of sophistication. The process is costly in terms of equipment and personnel 
required. Such analysis should not be undertaken by device users, unless spec- 
ificaliy required by contractual obligation to the. Government customer. The 
Government customer should review each RFP or other contractual vehicle, to 
determine whether or not the data yield from such failure analysis is worth the 
cost, in terms of failure mechanisms, and/or trends that may be revealed, prior 
to inclusion of the techniques, and release for bid/implementation of the con- 
tractual vehicle. 

3.2.3.U. Method $00k Screening Procedures : This method establishes the screen- 

ing procedures for total lot screening of microelectronic devices j to assist 
in achieving levels of quality and reliability commensurate with the intended 
application. Three (3) levels of screening rigor have been assigned arbitrar- 
ily , and these are described in MIL-HBK-217A Military Handbook for Standard- 
ization of Reliability Prediction. Over simplified, the levels are: 

A. Extreme Rigor 
B Rigorous 
C Minimal Screening 

Class B is to be used where no other class is prescribed. 

Screening Tests are tabularized together with the percentage level at which 
the screening is to be applied, for all three classes. 

Comment : The screening process has worked well since the inception of micro- 
electrnoic devices. It is an open question whether or not Class B screening 
has had better value to the industry than Class C, as the latter class has 
seldom been used. The process is costly, to implement, and the yield per lot 
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is directly dependent upon the screening level applied. It is not suggested 
that screening be discontinued, but rather than a uniform level of quality and 
reliability be established for the devices by genus, and a standard screening 
process be applied to all , dependent upon this figure of merit. 

3. 2. 3 . 5. Method $005.2 Qualification and Quality Conformance Procedures: 

This method establishes qualification, and quality conformance inspection 
procedures for microelectronic units, to assure that device ana lot quality 
conforms with the requirements of the applicable procurement documents. This 
•procedure involves producer, user, and ultimate user, the US Government .The 
method, cites: Tests to be performed as part of the inspection process, and 

tabularizes them into Groups and sub-groups. Test procedures are cited for 
both monolithic, and multi-chip microcircuits,: and requirements for sample 
selection, sample disposition, accelerated quality conformance testing, and 
substitution of tests/test sequences also are given. The requirements for bond 
strength and data reporting are specified, and finally, the exact contents of 
the procurement document are set forth. 

Comment: The procedure is usually implemented in association with the screening 
process procedure, and the costs are in direct proportion to the Lot Tolerance 
Percentage Defective (LTPP)._ The tighter the LTPD the greater the inspection 
costs,, and in turn, if the inspections are carried out by all three involved 
parties, producer, user, and Government, cost escalation is rapid. It is strong- 
ly recommended that the tests be carried out once only in as many cases as pos- 
sible," with test surveillance by both user, and Government QA representatives. 

3. 2. 3. 6. Method 5006 Limit Testing : This method stipulates means for establish- 

ing or. evaluating the Maximum capabilities of microelectronic devices, includ- 
ing such capabilities as absolute maximuM ratings (from which safe design limits 
may be derived), maximum stresses which may be applied in screening or testing 
without causing degradation, and sensitivity to particular screening and test- 
ing stresses, and the associated modes or mechanisms of failure. The types of 
test are included as are the sample sizes required for each test. The method 
recognizes that the tests are destructive in nature, time consuming, and costly. 
The testing is not expected to be conducted on all procurements, but is advocated 
for testing new types, and for the assurance of high device reliability. 

Comment: LMSC strongly recommends that this type of testing be conducted only 
on new types of devices, and only to the extent necessary to establish limits 
on critical parameters. The method itself is well designed as a procedure, and 
considers all essential criteria. 
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U»0i Cost Impact of MIL-STD-883A : Early in this EM, in paragraph 1.3, reference 
was made to the fact that the application of MIL-STD-883 drove program costs 
approx 1 7.5%. It was inferred that 1.3-2. 3$ of program cost might be saved by 
modification of both MIL-STD-883A, and MIL-STD-810B. The high program cost 
figure attributed to the application of MIL-3TD-883A came about for several 
reasons as listed below: 

1. The two programs for which any microcircuit cost data were available 
■ for study were MVM-73, and AE (Atmosphere Explorer). 

2. These programs are among the very first to avail themselves of on- 
. board equipment using MSI and LSI (Medium, Scale and Large Scale 

Integration). 1 

3. The on-board equipment employing these techniques, and the microcir- 
uits thus involved was of relatively new design. 

I4. . At the time of spacecraft design, manufacture, integration and testing, 
microelectronics technology was expanding and being refined. Similarly 
the; MIL-STD was being evolved in terms of test methodology, device 
test criteria, acceptance criteria, LTPD levels, et al. 

3. In 1971 and 1972, and, even earlier in 1969 and 70 considerable trouble 
was being experienced in the aerospace industry with microelectronic 
- devices. The yield per lot tested was low, and each successive test 
further reduced the lot size; lead bonding was a problem of major mag- 
nitude, with leads falling off as a result of improper processing and 
controls, and long delays were being experienced in obtaining given 
devices from many device families. As a result device costs were 
< escalating rapidly. 

6. As an outcome of the conditions enumerated in paragraph 3, each device 
lot was subjected to extensive testing by the device manufacturers, 
the device users at the primary equipment level, and the primary equip- 
ment integrators at the subsystem and system level. Further evidential 
tests were required for device type certification by US Government QA 
representation. 

7. Test programs may have been too rigorous, as the AE contractor reported 
a yield of less than k®% on several device lots tested. 

All of the above factors combined resulted in the costs ascribed to these prog- 
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rams, and finally the value of 7 S% TPC was an estimate based on data of some 
dubiety. -The -real figure is unknown seemingly to anyone - , and may be less than 
the one deduced. It is believed that even the curtailment of one step in the 
multiple replications of testing would account for the savings value cited. 

5«Q Specific Recommendations : LMSC believes that adoption of the following 

recommendations will materially reduce the high cost of environmental testing 
at both the device, and the system level: 

o. Review and combine the two Stds into one conjoint NASA standard 
to cover all aspects of environmental testing at all hardware 
levels . 

o Specify in the RFP, or other contractual document, exactly what 
testing, at what levels the Government expects to see. Make 
the contractor's test plan the binding document for what test- 
ing is to be done, at what time, and to what degree of test 
replication. 

o Standardize device tests as much as possible as to methods, 
replications, sample sizes, parameters, et al. The present 
test methods. and standards tend to be too general, 
o Minimize test replications to the maximum extent possible, 

and in all cases where a test at the next higher assembly level 
will suffice to demonstrate merit, use this method 'to cur- 
tail ,the test load. 

- - • o MLnimjize the amount of test data reporting, and test document- 
ation required at all program levels, 
o Assure that QA and Reliability requirements specified are 
adequate for the program, not the best possible, as is the 
current practice. 

o Standardize the number of devices available for a program, and 
do not design new ones merely for the sake of doing so* 

Finally, until such time as the present MIL STDs are re-written, and re-issued, 
assure that the documents listed as controlling the overall test process are 
the latest issue possible, and a list of tests not-required (exceptions list) 
is appended to each purchase order, particularly at the device level. : 


Lockheed Missiles & Space Company 
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This analysis is presence m 

general overview of the conception, prosecution, redirection, and accomplish* 

of the task; whereas the second, Part 2, is a detailed technical discussion o 

the subject. 

Part 1 

I.! Task Ob.ieotives: The task was undertaken as the fifth (5th) of a series ^ 

of technical analyses, each a sub-task of Study Task .2. As such the task had 

two (2) objectives: 

(1) Generation of a "New-Look" Standardized WBS 

(2) Generation of a "Users' Handbook" to assist with the 
implementation of the WBS for future spacecraft programs. 

A preliminary concept was prepared and documented, and a review was made by 
NASA and LMSC of this concept. It was the consensus of opinion that ue 
limited period of the study, the two objectives were unattainable. _ 

Modified objectives were proposed by IBSC, and NASA concurrence was obtained 

for the modified objectives which were: ■ _ 

(a) Analyse and document the merits, and problems implicit m 
the development and implementation of a Standardized UBS. 

(b) Establish means for tracking/recording data to evaluate 

the cost impacts of Low Cost Practices using any type of WBS. 
1 . 2 . Task Background : Identification and quantification of high cost-driving 
practices, based upon historical programmatic, and cost data; from past and on- 
going unmanned space programs, were, and continue to be, the major objectives 
of the MSA Low Cost Program, Practices Study contract undertaken by MSC. 
Although many of the high cost practices of past programs have been quantified 
satisfactorily, many others have been identified as suspected high cost practice 
which could not be quantified from historical records. This lack of Adequate 
cost and programmatic data was evident in a data sampling of thirty nine (39) 
NASA St DoE historical programs. The data inadequacy revealed by this sampling 
process was verified by a survey of more than seventy (70) program managers, 
functional managers, and cost analysts, within NASA and the aerospace industry. 
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Fifty seven percent ( 57 %) of those interviewed believe that programmatic cost 
data existing within their organisations are not adequate , for quantifying imp- 
acts of program practices, nor for making comparisons between two or more pro- 
grams. Nineteen percent (19$) believed that their data were adequate, and the 
balance expressed no opinion. 

It appeared evident that that some NASA practices related to program WBS 
contributed directly to this lack of adequate cost records. Furthermore, trad- 
itional practices tended to preclude quantification of both high and low cost- 
driving practices in the several functional areas oft Systems Engineering & 
Integration, Flight Support Equipment, Ground Support Equipment, etc. 

The scarcity, and inadequacy of the historical data, and the fact that NASA is 
in the process of conversion to more cost effective methods of conducting near- 
term, and future space programs; gave rise to the conclusion that work should 
begin immediately on the development of a " NEW LOOK 11 WBS . This programmatic 
work/cost analysis tool, together with its associated implementation direct- 
ives for Low Cost Practices, will be of material benefit in prosecution of 

the NASA policy of total program cost effectiveness. 

• 

1,3, " NEW-LOOK" WBS - Capabilities Desired ; The desired capabilities inherent 
in a New Look WBS may be listed as follows: 

(a) To provide a candidate NASA STANDARDIZED WBS for future unmanned pro- 
grams, and thereby preclude the added costs of providing new, and different 
WBSs in several iterations, each tailored for each new program. 

(b) To provide a standard framework, with detailed formats for collecting, 
and tracking all program costs from several organizations at different locations 
(These include NASA centers, prime contractors, and subcontractors) in a man- 
ner that permits NASA to sum all program costs rapidly. 

(c) Provide an overall, capability for the comparison of the costs of 
various program elements, as well as total costs among programs; such that cost 
impacts of new low-cost practices under implementation, are reflected in detail. 

(d) Provide the capability for tracking and quantifying the cost impacts 
of low cost practices, such as the use of standardized spacecraft components, 
and low cost management practices, now being developed; under the direction of 
NASA./B3 Low Cost Systems Office (LCSO). 
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(e) To emphasize tracking/recording of all labor costs. { approximately 
65 % of each MSA budget dollar is spent for laborj and labor efforts are 
most directly impacted by low cost practices). 

Results Summary : The significant results of this study subtask are 

summarized as follows: 

o LMSC was unable to meet the initial program objectives of development, 
and documenting of a Standardized WBS and Users' Manual, within the time 
constraints of this study segment. 

o LMSC did develop a Standardized WBS Concept , believed to be adequate 
for implementation in unmanned spacecraft programs, subsequent to full develop- 
ment of the concept. 

o Also developed were concepts for the establishment of programmatic 
records which will provide a better history of program performance than do 
current record keeping practices. These new records will list low cost program 
practices used in work packages, and will summarize work package quantifiable 
data so that evaluation/quantification of the cost impact of program practices 
can be effected. (The conceptual records envisioned can be used with, or with- 
out a Standardized WBS) 

1,5 • Conclusions ; Several conclusions resulted from the Study subtask, and these 
may be listed as follows: 

o LMSC under-estimated the task of development of a Standardized WBS, and 
merely added to the complexity of the task by incorporating the require- 
ment for tracking/recording those details permitting the quantification 
of Low Cost Program Practices. 

o The prosecution of the task did allow the establishment of a basic con- 
cept, and firm bases for the development of a standardized WBS on some 
future occasion. 

o The basic concept established for standardization provides adequate 
flexibility within the work package framework envisioned; to faciliate 
and accommodate the small differences expected to exist among unmanned 
spacecraft programs. 

o Implementation of the Standardized WBS concept could save NASA several 
million dollars per annum. Traditionally, these monies have been spent 
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on the preparation of "tailored" new families of WBSs for each new 
space program. 

o Implementation of the concepts defined in this study, will provide 
improved visibility into program performance, as well as providing 
better programmatic records to facilitate cost evaluation. Further, 
current cost collection and reporting systems in use by NASA, will 
not be affected significantly as regards existing forms and procedures, 
o Cost effective development of the concepts set forth in this study will 
require the participation of NASA personnel on a day-to-day basis. 

Such participation will facilitate the adoption of incremental decis- 
ions, and definitions, in terms of modus operandi, and problem solut- 
ions. 

1»6. Recommendations : LMSC has formulated several recommendations as outcomes 

of this subtask of the study. These are: 

o That efforts be continued to develop and implement a Standardized 
WBS family for NASA/ Contractor use on unmanned spacecraft programs, 
o That a small team of NASA and Industry specialists be established. 

Such a team would be engaged in full-time development and implement- 
ation of an integrated family of WBSs, to cover all NASA and aerospace 
contractor effort on future unmanned spacecraft programs. 

Part 2 . Detailed Technical Discussion : 

2.1. Technical Overview : This section of the EM sets forth the premises upon 
which this subtask was formulated, the work actually accomplished, and the rat- 
ionale which was responsible for the abandonment of the original task object- 
ives, and the adoption of those which were implemented during the course of the 
analysis. 

2.2. Assumptions : Listed hereunder are the assumptions which underlay the orig- 
inal subtask concept: 

(a) NASA historical records are inadequate for the quantification of the 
cost impacts of many high cost practices applied to past programs. The same 
inadequacies prohibit the comparison pro gram -to -program of the costs and prac- 
tices which have been used. 
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(b) Historical cost records have proved to be inadequate for the 
cost-effective real-time management of programs. 

(c) NASA practices in the areas of Program Work Breakdown Struct- 
ures (FWBS) and Contract Work Breakdown Structures (CWBS) 
contribute directly to the lack of adequate historical cost 
and programmatic records. 

(d) These shortcomings could be eliminated by modifying WBS practices 
in such a manner that the size, complexity, and proliferation 

of WBSs could be reduced to some standard minimum. 

(e) Traditional NASA practice has been to fund the tailoring of 

a separate WBS for each program undertaken. Much of the funds 
spent for such a practice may be saved by the development of a 
Standardized WBS applicable to the majority of future unmanned 
spacecraft programs. 

(f) It is desirable to log the implementation, and potential impacts 
of low cost program practices, in all future program records, 
thus assuring that such records are available for post-program 
cost/performance analyses/evaluations . 

(g) Methods developed for tracking costs of program practices should 
be compatible with existing contractor cost-reporting systems, 
such as the USAF validated CSCSC (Cost & Schedule Control System) 
in use at most aerospace contractors' facilities. 

2.3. Analysis Approach ; The approach adopted for the analysis was based on 
both the assumptions made, and the desire to incorporate those capabilities 
and characteristics considered essential. A multi-step analysis was undertaken, 
and the several steps are listed as follows: 

(a) Analyse NASA and DoD Guidelines & Control Documentation related 
to WBS formulation, implementation, and use? to assure compatib- 
ility of the potential NEW-LOOK WBS with both the intent of 
such documents, and the NASA Agency-Wide coding system. 

(b) Lay out a preliminary concept for scoping the overall task at 
the program level, and indicate the method of incorporating the 
desired capabilities. 

(c) Analyse CWBSs from both historical and on-going programs to 
select, and incorporate their best features into the New Look 
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concept, while avoiding apparent errors & shortcomings, 

(d) Establish a preferred CV/BS concept for proposal as the candid- 
ate Standardized WBS for unmanned spacecraft programs, 

(e) Prepare a Users 1 Manual, including a WBS dictionary, to support 
the preferred CWBS concept. 

(f) Consult with LMSC financial management personnel to evaluate the 
implementation compatibility of the evolving WBS concept, with 
the LMSC government-validated CSCSC program. 

2 ,ij. Research, and Compatibility Analysis ; NASA and BoD guideline and control 
documentation concerning WBSs was reviewed and extensively researched to 
determine what methods were reccommended for formulation and ultimate use. 
Outcomes of this research, and their pertinence to this analysis task are 
listed as follows: 

(a) Five underlying reasons for the NASA decision to report costs 
at the spacecraft and system level only, are : 

(i) This summary level was considered adequate for 
program management in most cases. 

(ii) The level selected for cost reporting must be 
contractually binding. 

(iii) Lower levels are not made contractually binding, 
and they are uncontrolled by NASA, This permits 
flexibility for the contractor to make changes, and 
obviates the necessity for contract changes whenever 
a lower level WBS element is changed, 

(iv) Lower level WBS elements dealing with "program crit- . 
ical items" are made contractually binding on a 
"per case", or an "as needed" basis only. 

(v) To avoid the costs incurred in reporting at lower 

levels, and to avoid also, a multiplicity of contract 
changes. 

Election by NASA of the listed options has contributed to development of 
inadequate programmatic cost records, when these are considered from the 
standpoint of quantifying the cost impact of program practices. 
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The exercise of such options also may cause conflict between continued use 
of current WBS practices (which provide considerable program flexibility, 
but produce questionable historical records) and use of the proposed LMSC 
WBS practices reccommended as alternates. These alternate practices may impose 
restrictions upon program flexibility, but will produce better program records. 
Such records can only improve cost visibility for future spacecraft programs. 
There may be a compromise possible between established NASA WBS practices, 
elected from among the options listed, and the LMSC proposed alternate practices. 
If such a compromise is made it may prove of greater benefit to NASA than 
either the established practices or the proposed alternates. 

(b) Requirements already exist for the development of separate WBSs for 
not only each NASA field center, but also for each prime or associate 
contractor. Nothing is mentioned in the literature research about these 
WBS versions being developed within some common framework. A common frame- 
work would permit not only rapid NASA summation of all program costs, but also 
the easy identification of duplication of effort. 

(c) Requirements for providing the capability to track and report costs down 
to the work-package level already exist; although the option to use this 
capability is seldom exercised. This finding is significant with respect to 
tracking of program practices as, tracking cost impacts of standardized compon- 
ents, must start at the component level; and the tracking of cost impacts of 
management practices must start at the work-package level. These basic require- 
ments compell the conclusion that tracking of the cost impacts in a literal 
sense would be very expensive, if not prohibitive, in terms of time, dollar 
expenditure, and personnel effort. Such a conclusion mandates an explanation 
of the LMSC use of the terms "track, tracking, and tracking/recording." 

As used in this report, these terms signify recording or providing a means 
of traceability for analytical purposes . 

(d) Another potential conflict of the LMSC approach versus the approaches 
set forth in the DoD/NASA guideline documentation, arises from the LMSC 
proposal to incorporate the capability to report indirect labor into the WBS 
work-packages, in addition to the direct labor costs within NASA and the 
aerospace industry. Such reporting would facilitate the evaluation of program 
practices cost impacts on all labor activity . However, it is most unlikely 
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that this proposal will be implemented, because .of the conflict it would 
occasion with the many existing aerospace management, and business practices. 


2.5. Review of Existing WBS Examples : WBS examples, and their associated 
Dictionaries, were obtained from several programs, and subjected to critical 
review. Desirable features were extracted from these examples for incorporat- 
ion into the DMSC Preferred Conceptual WBS . This preferred concept was 
intended to constitute the selected candidate for Standardization, but was 
not completed, as completion was impossible within the constraints of the study 
schedule. The review included: 

o SESP 71-2 WBS (Proposal & Program) 
o D5CS III (Defense Satellite Communic- 
ation System) Proposal WBS 
o SUS (Solid Upper Stage for Shuttle) 


o MVM-73 Program WBs 
o LST Program WBS 
o SEASAT Proposal WBS 
o PELSS Proposal WBS 
(Precision Emitter 
Location Strike System) 


Proposal WBS 


2.6 System Descriptions Review : Selected system description documents were 

reviewed, as well as the LMSC Cost and Schedule Performance Reporting System, 
CASPER. The object of this review was to ensure compatibility of the candidate 
WBS with the capabilities of this mechanized reporting system. While this 
review was informative in a general sense, no firm conclusions as to compat- 
ibility of the WBS concept could be formulated. However, the information of 
general value is listed below: 

o Storage capacity, and capability for numerical manipulation 
of the computer is no limitation for the WBS concept, 
o The CASPER system can accommodate WBS requirements of 12-1^00 
time-card type charge numbers, and up to 3000 cost accounts, where 
a cost account is defined as an identifiable mechanism for summing 
elements of program costs, such as labor, etc. These limits are 
based on practical experience which has demonstrated that increasing 
the number of cost accounts, or charge numbers, beyond these limits 
tends to reduce the accuracy of the accounting process. Further, the 
cost of rechecking all the human effort involved in accurate coding 
of these numbers, increases dramatically, when the limits are exceeded 
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o let another practical limitation on the system is the quantity 
of lin e items of cost data that program managers, and cost anal- 
ysts can use effectively, at monthly, or quarterly intervals 
of program reporting. 

2.7. NEW-LOOK WBS Preliminary 'Concept - Total Program Level : Figure 2-1 
which appears on Page 10 , is extracted from NASA Document, MHB 9501. 2 A, 
and depicts the normal NASA Reporting Structure Matrix . From this foundation, 
an overview of the NEW-LOOK WBS Preliminary Concept was derived, showing 
WBS data handling at the Total Program Level. An illustration of this concept 
appears as Figure 2-2, Page 11 . Elements A, B, & C, represent the same items 
in both figures, and are self -explanatory. 

2,7.1. WBS Family ; Element D of Figure 2-2 advances the concept of using a 

standardized, set, or family of WBSs, and includes all organizations involved 

in a program; i.e. NASA installations, Prime-Contractors, Associate-Contractors, 

and Major Subcontractors. The study schedule did not permit the construction 

* 

of trial layouts demonstrating the method of configuring a WBS family to 
correlate all elements on a total program basis; however, standardization to 
the maximum extent possible appears to be a cost effective step. In accordance 
with existing NASA guidelines for WBS preparation, each organization involved 
is required to develop a new WBS tailored specifically for each new program. 
While the WBS is prepared new for each program, the type of work performed, 
and the organizations performing it, are usually repetitive program to program, 
for each functional division of labor. Repetition of the types of work perform- 
ed, is due to the practice of grouping a number of semi-fixed skills into a 
functional division of labor; and is due further to the repetitive nature of 
the types of end-items, and services required from these functional groups 
during the execution of every program. This repetition of tasks within each 
functional division of labor provides the basis from which a standardised set 
of work -packages for each labor division may be developed. Adoption of this 
concept would mean that standard forms could be developed for each labor divis- 
ion, and these could be used for all programs thereafter. The forms would 
provide : 

1. Descriptions of tasks performed routinely. 

2. Descriptions of program peculiar tasks (if any) 

3. Descriptions of task outputs; i.e. end-items and/or services 

U . Numerical data such as quantities of end-items, man-hours required. 



• WBS/PRODUCT-END ITEM ORIENTATION 
TO HARDWARE, PRODUCT, SERVICE 
OR TASK 


• ORGAN IZATION/FUNCTION-THE 

ORGANIZATIONAL ELEMENTS AND 
THE FUNCTIONS INVOLVED IN THE 
TOTAL CONTRACT EFFORT AND OR 
SPECIFIC WBS ITEM 


ELEMENTS OF COST-NATURAL 
CLASSIFICATIONS OF COST, 
ACCORDING TO THE NATURE 
OF THE EXPENSE INVOLVED. 


REPORTING STRUCTURE DATA MATRIX 

FIGURE 2-1 
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(with columns for initial cost estimates and final actuals) 

5 . Explanations of programmatic changes, which are causative of 
adjustments to any of the numerical data. 

This definition of a work package is different from that given in NHB $600.1 , 
but could be compatible with the DoD definition furnished in AFSCP/AFLCP 
173-5. Both definitions can be altered without any major impact on program 
cost accounting methods if NASA deems program economies essential. All of 
the foregoing premises suggested that the logical method for structuring 
a standardized WBS was to use the functional divisions of labor as key elem- 
ents. This method was adopted to structure the standardized MBS version 
which appears elsewhere in this EM. 

2.7.2. Checklist- Low Cost Practices : Element E of Figure 2-2 presents a 

standard checklist of Low Cost Practices (SCLCP). Figure 2-3 , appearing on 
Page 13 , provides an example of an SCLCP, whose purpose is to record all 
low cost program practices that NASA may wish to implement for future programs. 
The list is standardized, and controlled so that it may be adopted and used 
throughout NASA. NASA-wide adoption of a- standardized, controlled, list of 
practices, will assure that the list receives regular up-dating and revisions. 
For recording and tracking purposes, such a list would be attached to each work- 
package, to show the quantity and types of Low Cost Practices incorporated 
therein, and the degree of implementation intended. A further use of the list 
would be the promotion of the use of low cost practices, and to facilitate 
NASA/Contractor general program planning. 

2.7»ii* Work-Package Record Summary : Element E of Figure 2-2 illustrates a 
Work Package Record Summary to be used as a historical record-keeping method. 

The summary would record orientation, and numerical data pertinent to work 
task(s) described in each work package. Examples of data to be recorded are: 
o SOW numbers . 
o WBS element numbers. 

o Quantity and type of specifications imposed 

o Specific outputs/end-items & quantities of each expected to result 
from the work to be performed, 
o Task listings of work as part of the work packages 
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CHECKLIST » LOW-COST PRACTICES 
PROGRAM/ PROJECT i 

DEGRE1 

IMPLE1 

■ ■ — r 

£ OF 
MENTA1 

A 

'ION 

A 

PHASE: 

FULL! 

APPLIED 

PARTIALLY 

APPLIED 

NOT 

APPLIED 

LCP 

NO. 

PRACTICE 


RFP's 




1-1 

Complete Definition of Program Elements in Phase C/D 




1-2 

Preliminary of RFP by Contractor . ! 




1-3 

Use of RFP to Encourage Lower-Cost Response 




1-h 

Streamline Response Data Required by RFP 




1-5 

Minimum Acceptable Mission/System Performance Reqts. 


■ 

' ■ ■ 

1-6 

Lower-Cost Alternate Proposals 


: . . 


1-7 

Reduce Reqts. for Contractor Plans 





DOCUMENTATION 




3-1 

Use Low-Cost-Driving DRL’s 




3-2 

Documentation-Need Evaluation Form 




3-U 

Use Direct Contact in Lieu of Paperwork 




3-5 

Use of Contractor Internal Documentation 

A A 





y 

"V 


-v 


Approved : 


Program/Project Manager 

/§\ Explain on separate sheet for each LCP item so classified. 

/K Provide explicit reason and obtain deviation approval from NASA/HQ, 

Low Cost Systems Office, for any single LCP so classified. 


Figure 2-2 - SAMPLE STANDARD CHECKLIST FOR LOW COST PRACTICES 
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o Cost estimates for each task, and each total work package, 
o Functional divisions of labor performing each task. 

Data recorded on this summary, together with the check-list of Low Cost Pract- 
ices to be attached to it, constitute a mimimal practical set of historical 
data for the evaluation of cost impact of Low Cost Program Practices for 
future programs. Further, such data will provide for comparison between, and 
among, two or more programs. 

2.7.5 Discarded Concepts : In addition to those previously discussed, the init- 
ial concept contained at total program level, two ideas which were subsequently 
abandoned. The first was the use of a separate WBS devoted to labor costs, and 
emphasizing both their magnitude, and the impact of program pratices upon them. 
The idea was abandoned early in the analysis, because the increased paperwork 
proved to be prohibitive in the light of the value of the additional emphasis 
so gained. Computer print-outs isolating labor costs from other costs incurred, 
can be obtained from most cost reporting systems, at less expense. The other 
idea was to treat indirect costs in the same manner as the direct-labor cost 
elements are handled. Such treatment was expected to expedite the evaluation 
of the cost impacts of program practices on total program costs 5 not only more 
completely than is presently done, but also more accurately if possible. 

This idea was found to be impractical since the effort required to change 
indirect labor practices throughout the aerospace industry would be extremely 
costly, and time consuming. Moreover, the desired payoff in terns of practice 
quantification improvement might not prove significant, and realization of the 
NASA initial purpose in quantifying cost impacts of program practices can be 
effected without this idea being adopted. 

2.8 WBS Standardization Concepts - Spacecraft Project ; 

2.8.1 Cost Reporting at the Component Level ; Reporting costs at this level is 
desirable , as visibility of costing details will be improved. Further, the 
cost impacts of low cost program practices such as use of standard components 
can be quantified, when future programs are costed at this level. Most of the 
standardization concepts set forth are equally applicable to the reporting of 
subsystem, and system level costs; the traditional NASA mode of operation. 

Ik 
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Sample WBS layouts included in this EM include ideas obtained from review of 
W3Ss, and WBS dictionaries describing several historical, and on-going prog- 
rams. 

2.8.2. Major Observations & Interim Conclusions : 

(a) There is no "Best-Way" to organize a WBS. 

(b) Program-peculiar, "tailored" WBSs reviewed, all appear to have 
emphasized visibility in some areas of programmatic/costing activity, 
while providing "less than average visibility" in other areas. 

(c) All of the WBS examples reviewed, evidenced "Forced-Fit" in several 
areas} i.e. forcing work tasks into a particular block, which could 
have been fitted into any one of several other blocks, with equal logic 

(d) The WBS dictionaries reviewed exhibited "Forced Grouping", tnat is, 
combining a number of functions/tasks, so that the overall number of 
WBS elements, and presumably the subsequent number of work packages , 
can be kept to some manageable quantity. 

(e) It would appear, that by necessity, similar shortcomings must beset 
all standardized WBS concepts. 

(f) Standardized WBS concepts must exhibit equal visibility in all program 
areas described by the concepts. 

(g) The uniformity of programmatic visibility, and the cost comparison of 
two or more programs, could be improved -by: 

(i) Standardizing the manner in which "Forced-Fits" are made 

(ii) Standardizing the WBS level at which these compromises 
are applied. 

These two measures would not degrade the visibility of cost parameters 
within a single program. 

2.8.3. WBS Layouts (Partial) : Figures 2-h , k-5, & 2-6 which 'Appear on Pages 
17, 38, & 19 respectively, represent partial layouts included to facilitate 
discussion of W3S concepts. The figures are preliminary in nature, and do hot 
furnish information regarded as finalized for purposes of standardization. 
Figure 2-U depicts a conceptualized version of a spacecraft project level WES 
considered adequate in scope for a very large unmanned spacecraft project. 

The relationship is shown of the spacecraft project to the overall program, 

as well as the conventional WBS levels, and the major system level summation 
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Sample WBS layouts included in this EM include ideas obtained from review of 
WBSs, and WBS dictionaries describing several historical, and on-going prog- 


2.8.2. Major Observations & Interim Conclusions : 

(a) There is no "Best-Way 1 ' to organize a WBS. 

(b) Program-peculiar, "tailored" WBSs reviewed, all appear to have 
emphasized visibility in some areas of programmatic/costing activity, 
while providing "less than average visibility" in other areas. 

(c) All of the WBS examples reviewed, evidenced "Forced-Fit" in several 
areas j i.e. forcing work tasks into a particular block, which could 
have been fitted into any one of several other blocks, with equal ‘logic. 

(d) The WBS dictionaries reviewed exhibited "Forced Grouping", that is, 
combining a number of functions/tasks, so that the overall^ number of 
WBS elements, and presumably the subsequent number of work packages , 
can be kept to some manageable quantity. 

(e) It would appear, that by necessity, similar shortcomings must beset 
all standardized WBS concepts. 

(f) Standardized WBS concepts must exhibit equal visibility in all program 
areas described by the concepts. 

(g) The uniformity of programmatic visibility, andgthe cost comparison of 
two or more programs, could be improved^. by: 

(i) Standardizing the manner in which."Forced-Fits" are made 

(ii) Standardizing the WBS level at which-. these compromises 

are applied. \ 

These two measures would not degrade the visibility of cost parameters 
within a single program. yV 

2.8.3. WBS Layouts (Partial) : Figures 2 -It , 2-5, & 2-6 which Appear on Pages 
17, ]8, & 19 respectively, represent partial layouts included'- to facilitate 
discussion of WBS concepts. The figures are preliminary in nature, and do not 
furnish information regarded as finalized for purposes of standardization. 

Figure 2-U depicts a conceptualized version of a spacecraft project level WBS, 
considered adequate in scope for a very large unmanned spacecraft project*. \ 

The relationship is shown of the spacecraft project to the overall, program, 

as well as the conventional 'WBS levels, and the major - 'system level summation \ 
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blocks within the spacecraft project. System level blocks Ji.l. A- )i.T r are 
not discussed. Standardization concepts discussed under block h.2 Spacecraft 
Development are generally applicable to blocks lul & k.3. hence the discussion 
is confined to block U.2. and its details only. Subsystem level blocks h.2.1 
thru jj.2.10 are believed to be adequate for standardization in unmanned space- 
craft projects. Figure 2.$ . presents an extension of the Structures 4 Mechan- 
ical Suosystem block It. 2.1 , and Figure 2-6 presents an extension of the Space- 
craft Integration and Assembly Subsystem Level, Block U.2.10 . Components, and 
other component level end-items associated with the subsystem blocks are 
displayed as sub-tiered items of the subsystems. Major functional divisions of 
labor necessary to the development and/or integration of the subsystems, and 
sub-tiered elements of hardware/software, are displayed on a WBS level below 
the subsystem level. Specific tasks to be performed by each functional div- 
ision of labor, are displayed as sub-tiered items under each functional divis- 
ion of labor. As previously stated, the repetitive nature of the end-items, 
and services required in all unmanned space programs; and the repetitive nat- 
ure of all tasks to be performed to accomplish the end-item production, and 
service requirements; provide a logical basis for standardizing work-packages 
by functional division of labor. Work-package standardization, in turn, provides 
the basis for standardization of the WBS. 

The WBS element arrangement concepts shown in Figures 2-g and 2-6 allows 
tracking and reporting of costs at the component level, in addition to 
preserving the NASA practice of summarizing cost 3 at the subsystem level. 

The arrangment also allows direct labor costs to be summed readily, and report- 
ed not only by functional division of labor, but also as a separate total for 
all labor. Presenting and reporting labor costs in this manner, is expected to 
provide NASA with a set of improved labor cost data. Such a data set can be 
used to evaluate the cost impacts of low cost technical and management pract- 
ices to be implemented in future spacecraft projects. 

The key to successful standardized work-package development, and implementation, 
is the constraint to some manageable number, of the total number of end-items 
and work tasks which are to be tracked, and reported. It was impossible with- 
in the time constraints of this study, to complete the trade-off analyses 
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necessary to determine this manageable number of end-items, and work tasks; 
nor has it been possible to prepare exact definitions for them. LMSC is 
convinced that manageable quantities of end-items/tasks, and standard defin- 
itions for them can, and should, be developed. When such are developed they 
should be acceptable for both program management, and record keeping purposes. 
Acceptability could be realized by grouping generic families of end-items, 
services, and work tasks, into suitable categories; applicable to both program 
management, and record-keeping. To effect this end, considerably more time, 
and NASA/C ontract or interfacing will be required than was available during this 
study. A feasible approach would be to assign small teams of NASA and contract- 
or personnel to work in parallel, and in support of each other; so that trade- 
offs, and incremental decisions needed to establish acceptable work-packages, 
and their definitions, can be effected in a step-wise manner. During the study 
it was not possible to determine a "Near-Optimum" quantity of end-items, and 
work task categories. The difficulty of evolving a manageable number is out- 
lined in the following example: 

Example A : Approximately 5 ,000 cost accounts would be required to accommodate 
10 categories of labor under each of the 7 functional divisions of Figure 2-5 . 
Were these 10 categories of labor to be tracked against 10 individual end-items, 
such as 10 components, in each of the component level end— item categories 
shown under W3S block U.2.1 ; approximately 50,000 - 60,000 cost accounts would 
be required, should all nine (9) subsystem level blocks be included, (as shown 
m Figure 2.U ). Tracking 50,000 cost accounts exceeds the capability of most 
CSC SC types of cost reporting system, by an order of magnitude. Program manag- 
ers, and cost analysts, will not be able to manage, and/or make use of the 
20,000 - 25,000 line items of cost that would result. 

2-8.3 WBS Dictionaries : All WBS dictionaries reviewed used some method of 
"Force Grouping" work tasks, and the resultant output descriptions. Specific 
methods used for historical programs were numerous, and varied, and are un- 
important to this discussion. The non standard manner in which "Force Grouping" 
has been accomplished in NASA programs has precluded commonality of most 
details in historical records. This lack of commonality has, in turn, prevent- 
ed formulation of a logical basis for the comparison of costs, and cost effect- 
iveness, between two, or more programs. The concept of standardizing work- 
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package format, and content, can provide the necessary logical basis for 
cost comparison among programs, by standardisation of the means by which 
essential "Force Grouping" (and associated definitions) is accomplished for 
future programs. 
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1. INTRODUCTION 

Under Contract V-27S2 to "^TlsfsTtta tt^fad ideTfTcer- 

could result in s ig ni f icant 

savings in NASA space program costs. 

• .p +hP Tow— cost practices with NASA Program Managers, some have 
In preliminary review o. the low j n „ the prac tice 3 would increase the program 
evidenced a feeline that mpleme, ntovion of the prac- u.c. study . 

risks, particularly those involving maasnQn suoceBS P lcH -cost practice. 

SHi™ r&VZ ? aSlepKd riSf«lSnM5S LCPP-15 dated 18 Oct- 
ofe”^! "fss^LTof Risk’ Impact - Low-Cost Program Practices". 

Another subtask of the Lou 

Re"f Sta° required/desired ioTow later Quantification of the risk impact. 

By agreement with the Low Cost Systems Office, and “TvTcostlaS'vSTmod- 7 

experienced in obtaining ^"dardise ostVeporting system; the suhtask objective was 
ified Work Breakdown Structure and cost repo: Stature on methods of quantifying 

altered to comprise i^gilrTS^eseTT results of this su^y; the 

Sor^tioriy^ ^fTtomSA J a base for further investigations into risk 

quantification . 
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DEFINITIONS AND GLOSSARY OF TERMS 


a "Risk" - The probability that a planned event will not be attained within 
theTcons train ts of cost, schedule, or performance by following a specified 
course of action. Risk is the complement of the cumulative probability: 


R - 1 - P p (Ref 2 ) 


b. 


Ilisk Analysis - (l) A process by which uncertainty associated with research, 
developmentT” and production can be exposed , controlled , cr reduced through 
qualitative and quantitative methodology for purposes of decision making. 


(2) A procedure for transforming decision-making-under-uncertainty to decision 
making -under risk (Ref 1 ). 

c "Formal Risk Analysis"- Formal means a separate and documented effort, con- 
ducted with normally accepted scientific investigative criteria (Ref 4). 


d. Performance Risk - The probability that the planned hardware/software mission 
performance will not be obtained. 


e. Cost Risk - The probability that the actual costs of a program will grow and 
exceed those estimated or targeted for the program. Cost overrun probability. 

f Schedule (or Ti me) Risk - The probability that the planned schedule will be 
exceeded or deviated-from, which will result in increasing cost or not meeting 
a critical schedule milestone (s ) , 


"Target Uncertainity" - Associated with defining a need or required operational 

° capability and reducing that need to cost, schedule, and performance goals. 

(Ref 4) 

h. "Technical Uncertainty" - Can it be done at all, for any price: (a) Does, the 
technical solution lie" beyond the program time .and. cost constraints.; (b) is 
the solution beyond present technological capabilities? ( rt ef 2 ) 

i. "Internal Program Uncertainty" - Associated with selecting a particular man- 
agement approach and carrying it out. (Ref 2). 

,i. "Process Uncertainty" - Derived from influences external to the program itself, 
includes interservi.ee rivalry, national policy, budgeting considerations, 
Congressional activity. (Ref 4) 

k. PERT (Program Evaluation Review Technique) - A network analysis system, origin- 
ated for the Polaris program, which comprises graphical time-based representa- 
tion of prgram principal activities and incremental events with interrelation- 
ships among events shown by a network of- interconnecting lines. Used for show- 
ing schedule impacts throughout the program of delays in accomplishment of ary- 
one (or more) specific incremental activity. (Ref 1,3) 

l. RI3CA (Risk Information and Cost Analysis) - A network analyzer computer routine 
more sophisticated than PERT, Allows simultaneous analysis of time and cost. 

(Ref 1,3) 

m. VERT (Venture Evaluation and Review Technique) - Has all RISCA features plus 

more flexible types of node logic. Includes a performance variable. Permits 
several types of functional relationships between/among Time, Cost, Performance 
variables. (Ref 1). 3 
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1. SUMMARY 


1.1 General Discussion of Available Literature 

Listed in Section 5 , "References", are many documents which appear to have 
either a direct or indirect bearing on quantification of risk, it is believed 
that these documents represent a valid cross-section of the state-of-the-art. 
Principle reference only were selected from our local LMSC library and reviewed 
and excerpts taken to highlight the most recent information available. The sel- 
ected references were References (l), (2), and ( 3 ). A number of other referenced 
documents are in our library but were not reviewed again for this subtask effort. 
Reference (39)was obtained informally recently from the Huntsville office of the 
Planning Research Corp, 

Most of the data relate to DOD systems requirements for risk assessment/analysis . 
The best general reference document appears to Capt Herbert Bevelhymer's thesis 
paper. Ref (l). 

1.2 General Observations About Quantifying Program Risk 

a * Pata Related to Economic Risk 

The majority of historical documents on quantitative risk analysis relate 
to calculating the risk of investment of dollars into new-business ventures or 
the probability involved with human life expectancy such as insurance actuarial 
tables. Some of the statistics are based upon estimates; however, most are 
based upon fairly large amounts or quantities of actual statistics which have 
been consolidated and analyzed. 

These data are more-or-less readily available in text books on economic risk 
and probabilities. These general references are not included herein. 

b. M^st Program Risk Analysis Directed Toward Analysis of On-Going P r ograms 
In the several documents reviewed (DOD sources), the principal emphasis 
was on the analysis of on-going programs. The objective was to assess the trend 
of program toward lesser risk or greater risk by review of actual program data 
(costs, schedule, and technical accomplishment) , versus initial estimates or 
program targets. Also, some analysis was devoted to comparative risks when 
judging whether a change should be implemented into an on-going program (usually 
analysis of cost/time alteration as a result of change). 


c. Risk Tradeoffs in Terms of Hardware Reliability 

Some historical data treat technical or performance risk in terms of 
reliability of the hardware. The risk is defined as 1 - P r where P equals 
the reliability or probability of success. For this type of analysis, actual 
data (such as historical test results, failure rates, or similar) must be avail- 
able to identify the analysis as statistically valid. Otherwise, the analysis 
is based upon subjective data, or estimates. 

d. Little Data on Performance Risk 

Most of the data available Seals principally with cost and schedule risk 
parameters. Because of the large variations in parameters which may be used to 
derive performance risk, most have avoided looking into this latter area in any 
depth. Ref (39) is an exception; however, the analytical approach therein 
is hypothetical in nature and is based on personal and subjective estimates, 

■The purist statitician might dispute the general approaches taken in the abalysis 
on the basis that actual program data were not used. 
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«• Consideration of Risk Before the Program is Started and Applicability 
to Low-Cost Practice Assessment 

Throughout this assessment of state-of-the-art on program risk, we were 
looking for two things : 

(1) Risk analysis methodology, used to analyze an on-going program 
status, which might be modified to analyze a new program not 
yet undertaken 

(2) General risk analysis approaches which might he adapted to include 
analysis of low-cost practice impacts 


2. GENERAL CONCLUSIONS OF THE SURVEX 


2,1 It appears that there is no directly applicable existing methodology which 
can be applied to quantify the schedule time, cost, and performance risks of 
a new (unstarted) program. 


2,2 To assess risks in a quantitative manner, particularly performance risk, 
with a reasonable degree of statistical validity, sets of data are required: 

(l) A fairly well-defined design of the proposed new-program hardware. 

^2) A good estimate on the various elements of the new program, with 
detailed estimates down through the major component level of the 
Work Breakdown Structure. 

(3) historical data from previous programs, both "normal 11 and "low-cost". 
These data as a minimum should include : 

a * ferf ormance Requirements , at vehicle, subsystem, and component 
level. 


b * Schedule Data , preferably in a PERT-type or VERT -type network, 
c. Cost Data , at vehicle, subsystem, and component level. 

These data should be available on a "standardized" basis so that summations, 

means, and deviations can be readily determined and used as a comparison base 
for new-program risks. 


.vJJbJr,' o 1 ty f h ?‘ sto f lca J- data on actual low-cost programs (as outlined 
above in 2.2(3) seems to be almost totally lacking. If quantification of risk 
is to be pursued, ;nd based-Upon historical data, it appears that program data 
must be collected on a standardized and more complete basis than is being accom- 


3. RBQUIRSMENT5 FOR QUANTITATIVE ANALYSIS OF LOW-COST PRACTICES 

3*1 Historical Data Required for Formal Sta tistical Analysis 

, f bhan a subjective estimate is required for new-orogram risk analysis 
" a ° bual daba from Previous programs is essential. Specific and usable data on ' 
low-cost" programs is probably not available at this time. If "normal" practice 
^ ir attendant 'typical cost, time, and performance parameters, are used as 

sh^u= 1 If C ^ m ?+ rat ?' Ve hlstori ? al baseline j all new programs analyzed will possibly 
show a statistical increase in risk if thwir equivalent parameters are the result 
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of low-cost practice application. For example, deviations from the 11 normal" 
practice time, cost, or performance values could be interpreted (analytically) 
as increase in risk. 

It is therefore necessary to create comparative a set of actual data on "low-cost" 
programs, particularly where mission performance has been acceptable. 

3,2 Time Variants or Schedule Risk 

a. Analysis of schedule risks requires a PERT-type schedule network on each 
project to be evaluated. Historical projects require similar sehedule/time 
representation (and most NASA programs have not used PERT-type networks). 
Assuming both new— program and historical-program data can be made available, 
and assuming some degree of standardization to allow easier comparison, element- 
by-element; similarity ratings can be derived. 

b. The new-program elements could be compared with the baseline set (histor- 
ical) using either manual or automated scanning techniques. Deviations from the 
historical norm would be rated as a degree of risk. 

' c. The baseline reference times would require subdivisions, probably to the 
component level; for example, time to design a specific major component or a 
group of subsystem components. 

A problem foreseen with schedule network comparisons and summing to the total 
time is the degree to which explicit elements of work or activities may overlap. 
Figure 1 shows a hypothetical case wherein the design of compoents A, B, and C 
are identical for all three cases, but the overall schedule time for the sub- 
system group of components is quite different. 

Design A 

Design B 

Design C 

Overall 
Subsystem 
G r oup 

Case 1 Case 2 Case 3 

Figure 1 - Three Cases of Schedule Summation Variation 



3,3 Cost Variants or Cost Risk 

Analysis of costs requires a common base of comparison. Because of the wide 
variation of subsystems performing a similar function, it appears that the cost 
breakouts to the component level is required to allow comparison of costs with 
any degree of validity. Summing of costs could be accomplished at subsystem 
and vehicle level (with added cost elements for integration, program management, 
etc. ) . 

Assuming that cost data were available at the component level on both historical 
and new programs, complexity, operational environment, MTBF, and other factors 
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could be applied and comparisons of cost derived. The deviation of the new- 
prcgram component cost from the baseline historical could be a measure of cost 
risk (probable co3t overrun or underrun), 

A requirement for collection of costs on a uniform basis must be applied: a 
standardized cost recording system appears mandatory. Inclusion or exclusion 
of items such as component testing cost or component test equipment or tooling 
costs could cause major variations (frequently tooling, test equipment, and 

testing costs are summed m subsystem or general cost blocks and not included 
in component summed cost). 

There are other cost-impacting parameters (such as inflation, labor-rate vari- 
entities Whl0h arS non “ risk " related and must he handled as separately-normalized 

3.U Performance Risk 

This risk characteristic is the most difficult of the three risk areas to 
be quantified m terms which will allow comparison of a baseline (historical) 
with a new-program requirement, ' 

Performance characteristics have not yet been standardized: also, the actual 
^ °" histori = al P">*ra" 1. frequently nit S 

raisslon requirements. Many of the requirements have been 
met grtially , but apparently adequate enough to rate the program as successful. 

leJels- a iehicL WO snh, S ^ ra advl ^ ble define Performance requirements at three 
if i i i subs y stera » and component. Hopefully, a direct relationshio could 

e estabiished for each performance parameter at all three levels For examnle 
pointing accuracy could be related to a Star Trader frnmnnnfl+t V 1 e:cam P-f e » 
Guidance 4 Central Subsystem, and at 4 tlLl "ehicliTJeT ’ NaVleatl ° n 

it a ppears that assignment of risk to a level of performance misht hava to ho 

niSSTo? F * rf ° rman!!e rlSk ' In ^ » fairly detailed desi^ defi- 

ration of the new-program component would be needed to allow a rigorous statis- 
tical performance risk assessment. rigorous statis- 

3.5 Combined Time/Gost/Performance Risk 

Rp la ?^ at ;° nSh i PS be established or less readily between time and cost 
Relationship of performance and cost is more difficult Thor-o -is not 

a di^ernibie statistical relationship batmen pfrSLJ Ld Uni SlerrlSL 
all three risk parameters is necessary to obtain and nvoraii . , ... 

theoretically poesible but till require epecUic de^L^ 5 ™ ^ thlS iS 
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k* HIGHLIGHTS OF LITERATURE .REVIEWED ON QUANTITATIVE RISK ANALYSIS /ASSESSMENT 
k.l General Approach to Quantitative Risk Analysis (Ref 1) 

U.1.1 Objectives 


, t ^ Devise a technique to use, for quantitative ti me/cost risk assessment, 
data that a program office normally receives from a contractor under CSCS re- " 
porting requirements . 

( ,, Heduce the high cost of currently-accepted quantitative risk assessment 
'gathering of special data and lengthy analysis). 


c. Methodology directed toward continuously assessing risk during develop- 
ment phase of a weapon system, ” ° * 

U.1.2 Summary of Approach 

a. Formal risk analysis has become a required part of the weapons acquisi- 
tion process since 1969. 


b. Many methods of quantitative risk assessment require extensive data 

collection solely for risk purposes. Frequent assessments therefore become 
costly. 

C * Ref (l) proposes a risk assessment methodology that uses contractor- 
reported data extracted from -the standard Cost Performance Report and the 
Schedule Status Report . — 


d. A graphical network of a project can be constructre using the symbol- 
ogy and logic available with the Venture Evaluation and Review Technique (VERT) 

^™ puter routine . Each network arc can be made to correspond with a contract 
Wdo element. 


. S * - The + P aran, ®te- PS °f 8ach arc's time and cost distribution can be deter- 
mined using the contractor's monthly revised estimates for WBS element costs 
and completion dates. 

f . A test application on a weapons system currently in full-scale develop- 

. The reSults were inconclusive, but did show promise 
for further application of the methodology. 


U.1.3 Hypothetical Risk Profile 

v ■u^-f^ re J 2 sbows a Hypothetical approach to pictorially showing risk A 
probability density function (pdf) is graphed as a Beta distribution, converted 
to a cumulative distribution function (cdf), and then the complement of the cdf 
is graphed as tne h st - u 0118 CQI 


;isk profile. The risk profile provides an indication of 
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U.l.U Emphasis on Network Simulation 

Many techniques for quantitative analysis have been proposed; the most ex- 
tensive treatment is given to various network simulation approaches, simulation . 
techniques are an advantage when actual data are not available, and permits 
conditional probabilities to be used. 


14 , 1,5 Subjective Estimation Required . 

For PERT and other network techniques, the mean and variance of the individ- 
ual distributions must be estimated to be quantitatively useful. Two serious 
obstacles exist to estimating for a development project: 

a. Usually there are no data from which quantities can be estimated; 
thus estimates must be subjective. 

b* It is very difficult to form a subjective estimate of the activity time. 


14,2 Simulation Methods in Risk Analysis 


1;.2.1 General Applications of Simulation Analysis (Ref 3) 

a. Difficulty associated with sensivity analysis was chiefly that no esti- 
mate of likelihood of occurrence of various system costs could be obtained, 
jtfonte Carlo methods (among others) can be employed to generate the likelihood 
profile of total system costs where iu is possible to estimate the profile for 
individual cost factors. 


b. Time to project completion, cost of project , and performance of 
resulting systems usually depend upon a number of input random variables. If 
one is able to impute probability distribution to each input variable, the 
problem still remains to find the output distribution. Figure 3 is a hypothet- 
ical graphical illustration of the general case wherein: input variables are 
costs associated with (l) physical subsystems and (2) changes in system config- 
uration (changes in requirements). 



Figure 3 - Graphical Representation of System Cost Risk Analysis 
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U.2,2 Some Quantitative Measures of Uncertainty (Ref 3) 

a. Probability of meeting or failing to meet specific goals of project 
performance, cost, or completion time. 

b. Expected value of a slippage of any of the time, cost, or performance 
factors . 


c. Variance or standard deviations of time, cost, performance. 

U.2.3 Monte Carlo Simulation Csing PERT 

a * Monte C a rlo simulations can be applied to PERT networks. Each trial of 
the simulation yields a "realization" of the network with a fixed value for each 
of the activity durations. Project duration equals the sum of the times on the 
critical path. After simulation has been conducted a large number of trials 
(10,000 not uncommon), the estimated mean project duration is computed as the 
arithmetic average of simulated durations for all trials. 

b. Each arc time (between network nodes) has a pessimistic, most-likely, 
and optimistic estimate. A 200-activity network and 10,000 trials would require 
20 minutes on an IBM ?0 9b computer, 2.5 minutes on an 1108, PERT can be extremely 
time-consuming for large networks] nevertheless, outlook for the future of planning 
R&D projects includes much more in the way of computer-assisted PERT simulations 
than exists today (1971). 


c. A typical output 
illustrated in Figure h. 


Project Direct 
Cost 


of the simulation run may show a distribution as 


•Network contains critical 
path with all activities 
at "crash" times 


"All activities 
at "normal" times 


Project Completion Time 


Figure U - Project Cost vs Project Time 


U.2.3 Use of RISCA Computer Routine for Simulation 

a * (Ref 1) RISCA (Risk Information and Cost Analysis) is a network analyzer 
computer routine more sophisticated than PERT; 

(1) Allows simultaneous, analysis of time and cost. Treats an activity 
cost as a linear function of activity (may be unacceptable in many applications] 
costs may require separate processing), 

(2) Incorporates Monte Carlo processing and provides output time and 
cost distributions for every terminal node of the network. 

(3) Permits greater flexibility in characterizing activity probability 
distribution] it does not limit inputs to Beta distributions. 
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b. (Sef 3) A user-interactive computer program, RI3CA , that facilitates 
snslysis of systems that are represented by a general class of network structures 
can be used. It provides the capability to construct desirable net representation 
of this system. Then it can run Monte Carlo simulations of the network. 

based upon the output statistics from a run (for example, time, cost, and fre- 
quency distributions), the user can modify his net to evaluate system design. 

U.2,5 Use of VERT Simulation 

U.2.5.1 Bevelhymer's Approach (Ref 1) 

a. VERT (Venture Evaluation and Review Technique) incorporates more 
flexible types of node logic than RISCA. Permits a more accurate modelling of 
real-world situations. 

b. VERT also includes a performance variable ; the variable can be selected 
by the user. Permits several types of functional relationships between/among 
time, cost, performance. 


c * Different critical path criteria, other than time, can be selected. 

A cost critical path or a critical path determined by weighted comparison of 2 or 
3 variables can be specified. 

d, VERT can discount costs, adding present-value cost dimension. 

e. One possible difficulty with VERT: the performance measure specified 

must be additive (just like time and cost) from node to node on the network. 
Development projects, consisting of aggregation of subsystems and standard 
performance parameters among subsystems , are not homogenous . For example , 
thrust^ and specific fuel consumption for propulsion subsystem cannot be added 
diredtly to drag and weight for an airplane subsystem. ~~™ " “ ’ 

U.2.S.2 Attempt to Use VERT on A-10 Aircraft Program (Ref 2) 


a. Methodology 


(l) Consisted of constructing a realistic summary network of the A-10 

Full Scale Development Project up to the time of 2nd DT&E aireraft mating to the 
30 mm gun. 


. _ network was based upon WBS elements to afford commri ann with 

the Cost Performance Report data. . ” 

„ . ^ 5 ata were based on Principles of subjective probability, estimates 

uo be obtained from contractor personnel. 

(U) . Estimates were to be fitted to a triangular distribution to obtain 
the required time and cost probability density functions for each activity in the 
network. J 


(5) The risk profiles for individual activities were to be computed 
and aggregated using ,the VdRT simulation routine to obtain overall risk profiles 
for schedule and cost. 


b. Major Problems 

(D Unable to find experts within contractor organization who could 
provide aggregate time and cost estimates for the entire WBS activity Crux of 
problem lies in the relationship between the WBS and the functional structure. 

(2) In order to determine how far along one activity must be before 
another can begin, one would have to network the project at the work package 

n ~ ~ “ ~ 
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level There are 3000 work packages in the A-10 airaraft structure program alone. 
* (3) Biased time/cost estimates from the functional managers was also 

a problem. 

c. Res ults . , , ... 

Unable to construct a WBS-based network that was consistent with the 

requirements of the basic methodology. The methodology was abandoned as imprac- 
tical. 

U.2.6 Range of Esti mates Required (Ref 2) , 

The major data premise for risk analysis is that cost, time, and periorm- 

ance parameters are realization of random variables. One must obtain a ran ge 
of estimates for each estimate. 

14 , 2.7 Cost and Time Summations (Ref 2) 

a. All arc costs an the network are summed for total system cost. 

b. Only those times on the network critical paths or near-critical paths 
contribute to the total system development time. 

U.2.8 Diff iculty With Performance Parameters (Ref 2) 

ormance risk is the most ill -defined and perplexing parameter. The 
many factors that could be used in one program depend on weapons, threat, mission, 
and other parameters. 

Performance parameters are not homogenous and hence do not readily lend themselves 
to an additive scheme like network simulation. For example, the A-10 aircraft 
specified general performance requirements as high payload capacity, small turn 
radius, long loiter time. There are no mathematical transformations which allow 
additive accumulation of a performance parameter that simultansously represents 
ordnance capacity, radial acceleration, and time -on-target achievement levels. 

U,3 Techniques for Determination of Distribution of Performance Factors (Ref 3) 

a. Analytical methods may be difficult or impossible. 

b. Numerical methods range from (l) numerical integration to (2) probability 
models based upon discrete approximations of continuous distributions. 

c. Simulation probably will require large quantities of runs to assure sig- 
nif i cancel 

d. Quantification of the likelihood of meeting the technical objectives 

may be exceedingly dTfficult; some cases may be relatively simple. Some techni- 
cal objectives are of an all or nothing variety; others represent cases where . 
some departure from the specification requirement is not catastrophi c (attention 
was focused upon the latter case), 

e. Performance variables are functionally related to: (l) design variables, 

(2) operating variables, and (3) environmental variables. For example, payload 
of a helicopter is affected by: 

o Air density - environmental variable 

o Rotor lift coeff. - design variable 
o Operating weight - operational variable 
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f. Experts were asked to estimate performance characteristics and to . approx - 
imate distribution profile ; the distribution profile was cataloged as uniform, 
triangular, or constant ^3ee Figure 

In 

Uniform Triangular Constant 



Figure 5 - Distribution Profiles of 

Performance Characteristics 

g. The next step included the determination of joint probability that all 
performance specifications would be exceeded. In the case of the helicopter 
rotor problem, the Risk Analysis Team (RAT) concluded that the program risk 
was very high (a low probability, O.U 25 to 0.125, that performance requirements 
would be exceeded). 

k.U Use of Actual Data in Lieu of Estimate 

a. Literature on applied risk analysis is highly weighted in favor of sub- 
jective assessment techniques to provide the necessary input probability distri- 
butions for a network model. If a methodology could be developed to use data 
currently provided by a contractor to estimate these inputs, it could afford an 
inexpensive method for assessing risk on a continuing basis. 

b. Collection of monthly WBS completion costs and time estimates can each be 
considered an independent, random sample whose size is increased by one each time 
a monthly report is received from the contractor. 

c. The range values for each WBS element must be determined for time and cost. 
Although not normally reported, data values could be provided at far lower cost 
than required currently for extensive subjective assessments. 

Sample Risk Profiles from Test Case Runs (Ref 1) 

Figure 6 shows sample risk profiles determined from test runs. They indi- 
cate the essential variations in profile resulting from sensitivity checks wherein 
the inputs are varied. 

It .6 Assessing Contractor Performance (Quantitative Risk Assessment) (Ref 2) 

U.6.1 Objectives 

Determine feasibility of a formal risk assessment methodology using sub- 
jectively-based probabilistic estimates as inputs for a c omputerized network 
simulation} apply the methodology to actual weapon system acquisition program 
environment, in this case, to the A -10 aircraft structure. This provides a 
method for forecasting contractor’s performance. 

The incentives for research study were notorious examples of cost overrun, 
schedule slippage, and performance degradation within major weapon system ac- 
quisition. In general, answer the question, "How do we go about assessing 
program risk?" ^3 
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Figure 6 - Risk Profiles 
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Development Cost 
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U.6.2 Observations 

a. Bevelhymer's work (Ref 1) is the only actual test case using network 
analysis; other have advocated it and produced hypothetical examples (Ref 23,33,3U) 
While not a network-based technique, Thomas (Ref 35 ) has successfully employed an 
accumulatiive approach using subjective inputs and producing probabilistic esti- 
mates of total risk in bidder’s replies to RFP. 

b. The technique which offers the most promise in quantitative risk assess- 
ment is a versatile , simulative approach using group assessment techniques , sub - 
jective probability , technolog ical forecasting, cost estimating, and other 
sources of input . 

c. Preponderance of risk analysis literature suggests use of subjectively 

obtained data; Bevelhymer (Ref 1) initially attempted simulation using data from 
contractor documents. He could not obtain such information solely from contractor 
data and was forced to resort to subjective estimates for range end-points for the 
cost variables. ~~ ™~ 

d. Most works on risk analysis are primarily directed toward recognition 
and assessment of technical uncertainties (Ref 31,33,36). 

e. because of the difficulty of identifying additive performance parameters, 
the performance aspect of risk assessment was disregarded and emphasis placed 
upon cost and time parameters. 

U.7 Examples of Performance Risk Analysis (Ref 3) 

The appendices of Ref (3) show examples of risk analysis: 

a « Appendix 1 - Risk Analysis of Armored Vehicle -Launched Bridge 

Evaluation of several alternate proposals and project stagings for a 90-foot 
segmented extendable bridge (was 50 feet) deployed by a tank. 

k* Appendix 2 - Micon-Air Defense Missile Warhead/Fuze Subsystem Performance 

Risk Analysis 

Assessment of probability of kill, mainly centered on fuze reliability. 

c * Appendix 3 - Helicopter Performance Risk Analysis (Hypothetical Example) 

Evaluation of likelihood of meeting performance specification requirements 
in light of serious mid-program trouble : serious dynamic instability problem at 
elevated speeds (delaying R&D efforts). Many performance variables; small subset 
of these included: 

o Maximum forward flight speed 
o Hover altitude 
o Maximum ferry usage 
o Endurance 
o Payload 
o Rate of climb 

U.8 Risk Estimating Techniques for Planetary Spacecraft Performance (Ref 39) 

a. The modelling was not tailored to use of actual program time, or cost 
data nor to program simulation techniques. Some of the parametric relationships 
shown which impact performance are rather interesting. In general, the tech- 
nique descrioed treats risk as one minus the probability of success (reliability). 
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b. Presents the results of a small exploratory study on development of a risk 
model for unmanned planetary mission, the Mars Orbiter/Lander 1973/75. 

I c. Deals primarily with risks involving: 

o Schedule 

o Number of spacecraft per launch 
o Number of launches 
o Impact of spacecraft sterilization 
o Level of combined system testing 

d. Risk was quantified independent . of cost. Schedule risk was included as 
a management risk under mission design. 

e. All data were based upon subjective estimates. The model was calibrated 
using data from the 196U Mariner IV mission and parametric estimates for the 
1973/75 Mars Orbiter/Lander. 

f. The mission risk summary was shovm as: 

Rjj “ 1 - (1 “ “ R SBd) ^ ■ R CSt) (1 " a SFo) 

where : 

Rjjq » mission design risk 

RsBq“ risk for subsystem design/development phase 
R CST* combined system testing risk 
R 3 F0* space flight operations risk 
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1.0 Historical Background : 

1957 the specification which has controlled the production of engineering 
drawings and lists procured by the Federal Government of the U.S.A. has been 
MIL- D- 1000. This document was generated for use by the DoD in order that military 
materiel and capital equipment drawings might be standardized to some degree, 
reproduced in a large number of legible copies, and used by any given manufacture 
under Government contract to produce military hardware. In the rapid build-up of 
the NASA as a Government agency responsible for the manned lunar series of missions, 
the relatively short interval of less than a decade, precluded development of a 
drawings and lists specification adequate for the needs of the aerospace industry. 
MIL-D-1000 was inherited by the NASA, and used to procure drawings. While the 
speciiication served as a regulating document, drawings produced under its control 
have not been low cost, and considerable confusion arises from the interpretation 
of the specification in terms of its requirements. 

1.1 MIL-D-1000; Implementation Difficulties : 

Aerospace Contractors attempting to comply with MIL-D-1000 when this document is in- 
voked by their contracts with Government agencies encounter several difficulties. 
These arise from the fact that MIL-D-1000 can be implemented in any one of three 
Forms, and in one, or several of 10 categories. Details of forms and categories 
are as follows: 

0 Form 1 requires: Full Military compliance/control 

o Form 2 requires: Partial Military Control 

o Form 3 requires: Drawings ,to Industry Standards with minimum military 

controls . 

The 10 categories involved are titled as follows, and cover all conceivable tyres 
of hardware and hardware service : 
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A Design Evaluation 
B Interface Control 
C Service Test 
D Logistic Support 
E Procurement (identical items) 

F Procurement (interchangeable items) 

G Installation 
H Maintenance 
I Government Manufacture 
J Interchangeability Control 

HINDOO defines all these categories, and states that the Go_t win deflne 

specify, per each present from a contractor; the Fom(s) and categories) 
under wmch the drawings are to be prepared. 

- The Subt ier Document Difficulty ; 

Should a Government agency invoke Fo™ 1 of MIL-B-1000 in procuring engineering dra„- 
xngs and lists, full appliance by a contractor quires that no less than 38 sub- 
tler documents must he ac q uired, scrutinised, and their regents reflected in the 
drawings ultimately produced, under any or all categories. Fore 3, which has been 
the usual For, invoked by the NASA looses by sub-tier, only 13 docents, which 

alleviates the complexity of drawing production to a degree. «here Form 3 is invoked, 

on ly b documents ars invohpH 

cub invoiced as sub-txer dependancles Th-i o u - n , 

P 163 ’ iJlls Form ^ which has the minimu 

ot Government control, reauires nnitr . 

q 1117 that tr ““8 P^r of a given standard be used, 

that document s/drawings be inspected for count -irfPTvM+ n , 

unt, identity and damage, and that the 

Government ’s rights to security he observed. The doc^ents/specificaticns invoked 
as sub-tier. dependaricies for the 3 Forts, are listed in Figure 1. As can be seen 
rom this figure. Form 3 gives almost total latitude to the contractor in the 
preparation of drawings destined for Government • use, whereas For, 2 still imposes 
some of the requirements. The curre , opinion within the NASA is that Form 1 is 
too costly. Form 2 still imposes too much control, while Form 3 imposes too little. 

arise from the imposition of Form 3 by contractual requirement, since 


MIL D 1000 op ^ifications 


Dependency L3 - ting .’All tiers 


-Federal Spec:: 


UU-P-00561. . . Paper , Tracing . 
CCC-C-531 ••• Cloth, Tracing 


Military Standards : 

Ml S'i‘D iUOA. , . ENGINEERING DRAWING PRACTICES (Cost Driver) 
MIL-STD-1U3 ••• Order , of precedence, specs, stds. 

MIL-STD-9 ...... Screw iThreads 

MIL-STD-12C . . . . Abbreviations 

MIL-STD-lit Architectijral Symbols 

MIL-STD-15 * . . * . Electrical Symbols 

MIL-STD-l? Mechanical Symbols 

MIL-STD-18 . , . . . Structural Symbols 

MIL-STD-29 Springs, Mechanical. 

MIL-STD-3U ..... Optical Elements 
HIL-STD-275 • • • . Frinted Wiring 
MIL-STD-806. * . . Logic Symbols 


industrial Standards: 


USAS BU6.1 -62 Surface texture, roughness, waviness & lay 

USAS Ylli.5 -66. Dimensioning & tolerancing for engrg dwgs 

USAS YlU .19-66 Electrical & Electronic Diagrams 

USAS Y 32.2-67 Graphic Symbols for Elec & Electronic diagrams 

USAS Y32. 16-65 Electrical ct Electronic Reference Designations 

AWS STD A2.0-58 Welding Symbols 

AV/S STD A2.2-58 Mon Destructive To sting -Welds. 


Mil-Speci 


MIL-D-5Ii 30..... Data, Engrg & Technical^ reproduction of: 
MIL-Q-9858A. . . . Quality Program Requirements 

MIL-M-9'368 Microfilming Engrg Dwgs 

MIL-I-[(5208, . . . Inspection System Requirements 
MIL-C-h5662 . . . . Calibration of Measurement 2c Test Equipment 


MIL HDBKS : 


Hit-1., i Federal -Supply Code ; Name to Code 

Federal Supply Code: Code to Marne 

Ho-l. .......... Federal Items Identification Guide 

H7 Mfrs Part 2c Dwg Numbering System 


Mil Manuals : 
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( 2 )( 3 ) 
( 2 ) 


(2, portions only) 

( 2 ) 


( 2 ) 

(2)(3> 

*( 2 ) 

( 2 ) ( 3 ) 
( 2 ) 


( 2 ) 

(2) 


5220.22M....... DoD Industrial Security Manual 

U120.3M. ...... . Standardization Policies, Procedures 2: Instructions. 


(2)(3) 


All the documents listed above are invoked by imposition of MIL- D-1Q00. Form -1. 

(2) denotes Form 2 only. 

(3) denotes Form 3 only. 


-* Only if Requ’ d by GDRL 


Figure 1 
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there can be wide variation in the merit of drawings produced to commercial/ 
industry standards or so it is believed within Government agencies. 

1.1.2. The Drafting Practices Manual (PPM) : 

Over the years since MIL-D-1000 was first issued, the aerospace industry has 
developed DPMs. This has been done as a matter of necessity to ensure contractual 
compliance, and to inject a measure of uniformity into technical drawings. 
Contractors have become accustomed to responding to MIL-D-1000, Forms 1 & 2 
and have costed their efforts accordingly. In actual practice, use of Form 1 
has been limited, as not only are the costs high, but compliance with all of the 
Form 1 requirements is a time consuming, and tedious process. Facility drawings, 
which must for a very long time, '& from the masters of which many copies must be 
made, have been made to the requirements of Form 1, as the legibility requirements 
are stringent. Form 3, with its broad latitude, has not been used often by either 
the DoD or the NASA; the customary Form invoked being Form 2. While the DPMs 
may vary to a minor degree, aerospace contractor to aerospace contractor, most 
of these manuals are very similar one to another, as they have been based upon 
Forms 1 & 2 requirements, particularly the provisions of MIL-STD-1C0A, Engineering 
Drawing Practices. To a large degree, the practices set forth in MIL-STD-100A 
have dictated what industrial practices for drawing production have become, since 
MI L-STD-100A is based upon accepted industry standards, and in many cases these 
are invoked in their entirety by the STD as dependency documents (See Fig 1 ) . 

In effect, the DPM produced by any given contractor is submitted for Govt, agency 
approval as part of the contractor certification process, qualifying him to 
do business with the U.S. Government. Should the DPM not meet with agency approval, 
certification of the contractor can be with-held until the standard of drafting 
practice has been raised to the required level. As an outcome of these practices, 
even commercial/industry standards are not the lowest possible cost by any means. 
The cost situation Is futher complicated by the fact that aerospace contractors, 
accustomed to doing business in the manner prescribed by MIL- STD -100 A, to the 
Form and category stipulated in MIL-D-1000, are unlikely to produce drawings 


which differ markedly from Form 2 standards, even though Form 3 is specified. 

1.1.3* Cost Comparisons : Researches into the relative costs of producing drawings 
and lists in compliance with the three forms of MIL-D-1000 indicate that if drawings 
are produced to some basic cost figure compatible with the minimal requirements of 
Form 3j these same drawings will cost approx h0% more if produced to Form 2 require- 
ments, and more if executed in full compliance with Form 1. Other data suggest 
that some U?/ of the costs to produce drawings to Form 1 requirements are incurred 
by the assurance processes involved in determining that the requirements have indeed 
been met. 

1.1. h The Least, Cost Approach : 

As has been stated previously, MIL-D-1000 Form 3 when imposed, does not necessarily 
assure the production of low cost drawings, particularly when the contractor prod- 
ucing, the drawings is well accustomed to operating within the aerospace milieu. 

The fact that most of the military standard requirements cited in MIL-D-1000, MIL 
STD-100A, etc, have been derived from Industry standards, serves merely to produce 
drawings in which a relatively standard symbology, and set of drafting practices/ 
techniques have been applied. For many drawings many of these techniques and prac- 
tices are no longer necessary, nor are the 10 categories of MIL-D-1000 essential 
to the production; of drawings which. the Government can use. The elaborate nomen- 
clature and numerology prescribed by MIL-STD-100A, came into wide use as an outcome 
of efforts to provide configuration management and control for aerospace programs. 

In the modern system of computer produced drawings and lists, such precautions are 

no longer necessary. Many drawings used in development phases of programs need not 
be produced using ruled lines. Frequently a free-hand rendition is sufficient provid- 
ed suitable dimensions are furnished, and the drawing is legible. Similarly, 10 categ- 
ories of drawings are not needed, since the drawings in most of the categories differ 
only in the particular symbology used. The optimum number of categories will be discus 
ed in subsequent paragraphs of this document. 
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To summarize the measures believed to be essential for Least-Cost-Specifications 
governing Engineering Drawings and Lists , the following set of gu i de lin es was 
developed: 

o Minimize the number of Drawing Categories, 
o Delete compliance with Military Standards and Specifications 
and substitute Industry Standards, using these as a reference and guide 
only. 

o Delete the requirements for Quality Assurance, with the exception of 
the inspection requirement for Count Identity and Damage, immediately 
prior to delivery. 

o Permit use of free-hand drawings in all cases where these will suffice, 
o Delete the requirements for reference to Military Specifications, 
dealing with tracing cloth and paper, as these are standard through- 
out the aerospace industry presently, 
o Delete the microfilming and reproduction requirements as a set of gen- 
al provisions. Most frequently these are unnecessary in today's tech- 
nology , since in the case of most NASA programs fourth generation 
microfilm and a very large number of copies of drawings are not requ- 
ired. 

1.2. Compiling the Drawing Specification : 

1*2.1. Approach : With any new specification, there is an appreciable time lapse after 
its generation, and prior to its formal release. This interval allows the document 
to be circulated, so that potential users can become familiar with its provisions, 
requirements, and mandates. There is usually a period also during which comments, 
and recommendations for improvement/finalization of the document are furnished to 
the generating/releasing agency by the users. Since this document is novel in con- 
cept in that it sets aside many of the established means, and procedures for produc- 
ing Government used drawings and lists, in favor of Low Cost metihods; a different 
formatting approach has been adopted. Where considered necessary, each paragraph of 
uhe specification which xollows, has one or more paragraphs of rationale. 


These rationale statements, explain the underlying need for the preceeding paragraph, 
and the reasons why is is believed to represent a Low Cost approach to the subject 
under discussion/specification. The statements are arranged in such a manner that 
should NASA elect to publish the specification for use, the rationale can be omitted 
without any further change to the text of the document. Similarly, the historical 
preamble, and all foregoing text can be omitted without jeopardizing the meaning 
and application of the body of the specification. With all extraneous material of 
this type removed, the document can be implemented as a clearly defined set of 
contractually binding requirements with which contractors producing drawings for 

Government use must comply. 


2.0. Specific Requirements,: 

2.1. Scope ! This specification prescribes the requirements for the preparation of 
engineering drawings and lists, and for application of Intended Use Categories 
for their acquisition. The requirements reflect the desire of the Government to 
purchase legible drawings produced at the lowest price commensurate with their 
adequacy for Government use/purposes. The term Engineering drawings as used in this 
specification, incldes engineering drawings and associated lists. 

2.2. Classification ! NASA will acquire engineering drawings in one or more of three 
Use categories. These categories are listed as follows: 
o Design Descriptive/ Disolosive. 
o Procurement. 

o Installation & Maintenance. 

2.2.1. Rationale : The three categories listed above replace the ten (10) categories 

of MTL-D-1000. In essence the appearance and symbology differ little among the 10 


caegories. The differences lie in the use to which the drawings will be put. Since 
even these differences are ones of use, rather than technique, Design Descriptive/ 
Disolosive drawings will be inclusive of Categories A, B, & C of M1L-D-1000. 


Procurement Drawings will include categories, D,E,F & J, as the information on such 
drawings is quite similar per category, whereas Installation and Maintenance drawings 
will include categories G & H. Note that there is no inclusion of category I, as 
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In most cases, NASA does not manufacture prime equipment. This category was intended 
primarily to apply to Government Manufacturing agencies such as arsenals, producing 
a miscellany of armaments. 

2.3. Applicable Documents ; 

2 . 3 .1. One (1) Government document is imposed as a requirement by this specification. 
However, in cases where contractors have not furnished, hardware, software, materials, 
and services for the NASAj the following stipulations apply: 

0 NASA will specify in the RFP that a Drafting Practices manual will be 
provided for NASA inspection and approval. In cases where such a document 
exists', it will be inspected by a NASA, or NASA designated government 
agency representative, technically qualifed, and empowered to approve the 
manual. 

0 In cases 'where the contractor does not possess a DPM, such a document will 
be prepared, and submitted to the NA S A for approval. Preparation will 
be undertaken using the following documents as a reference and guide: 

1. This specification 

2. MIL STD 100A, MIL-M- 9868 , MIL-D-5U80 (For titles see Pg.3.) 

3. Industrial Standards: 

USAS M|6.1-62 Surface Texture, Roughness, Waviness & Lay. 

USAS Yli * 5-73 Dimensioning & Tolerancing for Engrg Dwgs. 

USAS Ylii. 15-66 Electrical & Electronic Diagrams. 

USAS Y32.2 -67 Graphic Symbols for Elec & Electronic Diagrams 
USAS Y3 2.16-65 Electrical & Electronic Reference Designations 
AWS-STD A2.0-58 Welding Symbols 
AWS-STD A2.2-58 Non Dstructive Testing-Welds. 

2.3. 1.1, Govt. Doc'mt : DoD Industrial Security Manual. #5220. 22M (or NASA equivalents) 
This document will be specified only if work prepared is of a classified nature. In 
such cases, the document will be specified in the RFP, and also in the CIRL call-out. 
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2»h- Legibility & Reproducibility : Engineering drawings shall be of sufficient 

clarity that an original will reproduce not less than ten (10) legible copies 

whan used as the master for such processes as Xerography, diazo-process, and 

white-print. Further copies may be obtained beyond 10 by repeating the process 

as many iterations as is needful, as these processes do not degrade the master original 

materially. For large numbers of copies, a photo process using a negative-positive 

method shall be employed. Blue-line drawing copies are not acceptable as deliverable items. 

2.5. Drawing Methods : 

2.5.1. Delineation : Lines need not be ruled in all cases. For drawings prepared 

under category A of this specification freehand sketches are acceptable, provided 
that sufficient detail is presented to allow hardware to be made from such sketches. 
Clarity of each sketch made shall be preserved as per para. 2.1;., and dimensions 
shall be furnished to permit manufacture to essential tolerances. 

2.5. 1.1. Rationale : In engineering and development phases of programs, sketches are 
often sufficient to allow the manufacture of hardware whose function is to prove 
the viability of a design concept. It is NASA practice to purchase hardware rather 
than drawings, and in such situations the contractor is instructed to retain drawings 
on file. Sketches can effect an appreciable savings if used for all phases of a program 
except production manufacture and assembly. 

2.5.2. Lettering and Line Formulation : For most drawings, pencil drawn lines are 

adequate. Ink shall not be used except for special processes where ink is a part of 
the process. Line weights shall not be varied except where essential to point out 
detail, but weights shall be sufficient to permit legible, clear reproduction. 

Lettering shall be freehand pencil characters, of simple Roman Block capitals and 
of sufficient size to permit clarity and legibility under photo-reduction processes. 
Italics shall not be used, except to highlight important statements. 

2. 5. 2.1. Rationale : Lettering using lettering guides, templates, Leroy processes, 

and others require additional time. They do nothing for the drawing except add to 
the eye appeal. Drawings are a means to the production of operable hardware, and 
their artistic appeal is of no value in meeting this requirement. 
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An example of a special process requiring ink, is the generation of Master Layout 
drawings for Printed Circuit Board (PCB) production, where the process requires ink- 
ing on Mylar, or other suitable high-plasticity material. 

Automated Draf ting & Mechanical Aids : Use of computers and associated devices 

to produce drawings is permitted in those cases where prior cost trade offs clearly 
indicate a marked cost advantage over ma&t&vL methods. Variables to be considered in 
such a decision shall include, number of drawings of a given type to be produced, 
degree of standardization of such drawings, cost per unit drawing, inclusive of all 
program generation time, machine run time, etc., end use for produced drawings, and 
NASA stated requirements. 

Mechanical Aids; Use shall be made of mechanical aids such as "Paste-Ups", transpar- 
encies, decals, and other transerrable symbols and characters, when such use offers ■ 
a clear cost advantage over traditional methods.- Implicit in the decision to employ 

mechanical aids of this type (these types), is the end use for the drawings to be 
produced. 

2. 3. 3.1. Rationale : Machine produced drawings can be quite costly in terms of the 
set up and debugging costs of the programs which produce them. Similarly "Paste-ups" 
and other transferable media, are most useful when a set of drawings is to produce 
many copies in reduced sizes from an original. When instruction manuals, maintenance 
manuals, and other publications are to be produced, both computer methods, and mech- 
anical aids used to produce "camera-ready" masters, can furnish significant savings. 

Numbering System; The numbering system included in a contractor's DPM shall 
be employed for numbering -drawings and associated lists. Wherever practical, such a 
numbering system shall consist of not more than 10 primary digits, and not more than 
2 primary digits as dash numbers indicative of changes. The numbering system shall 
be as per MSA contractual instruction in the RPP or other document. Where a number- 
ing system for a given program set of drawings is to be instituted for the first time, 
contractos shall use MIL-STD-100A as a reference and guide (Chapter 3 Numbering, 

Coding & Identification). 

2.6. Drawing Detail & Views : 
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2.6.1. Drawing Detail : Multiple Detail dwgs shall be produced whenever feasible. 

The detail given shall be restricted only by the available space, and the clarity 
of the drawing. 

2. 6.1.1. Rationale : It is more cost effective, and less time consuming to provide 

as many details as feaible on a drawing rather than attach a number of sheets 
which furnish supporting detail. 

2.6.2. Supporting Drawings : Where supporting drawings on separate sheets become 

necessary, it shall be the practice to attach all supporting sheets to the top draw- 
ing, and number it as a package. The top drawing shall furnish the overall intelligence 
at the highest level practicable, and the sheets supporting this drawing shall furnish 
all required lower level, and ancillary data pertinent to the top drawing overall 
subject matter. 

2.6.3. Drawing Views ; Drawings shall be prepared using the minimum number of views 
commensurate with illustrating the detail of the device that the drawing depicts. 

In most cases, plan, and side elevations will suffice. 

2 . 6 .I 4 . Drawing Scale ; In all cases possible drawings shall be made to full scale, 
so that the subject delineated be represented as full size. Where full scale is not 
feasible, a scale shall be chosen that is as large as is feasible commensurate with 
sizing the drawing(s) for ease of handling, ease of storage, and maximum clarity. 

2.6.5. Engineering Drawing Lists : Where the generation of lists is necessary to 

furnish data on parts, etc., such lists shall be included on the face of the drawing 
as practicable. Where lists are too voluminous for such a practice, they shall be 
appended to the drawing as a practical means of assuring their trace ability/availability. 
Lists shall be simple typewritten documents on standard A size sheets, or shall be 
handwritten/lettered, if this is practical in the light of the amount of information 

to be included. 

2.6.6. Drawing Titling & Margins : Elaborate margins shall not be used on drawings, 
nor are pre-printed elaborate drawing sheets in various sizes required. Titles shall 
be handlettered in a simple title block, which shall contain as a minimum, the drawing 
title, drawing number, and the signatory information, of originator, and checker. 
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2 * 6 * 7, — - awlng Sign-Of f/Apnroval : As a minimum, drawings shall be signed off by: 

(a) The originator 

(b) The checker 

(c) The designer approving the drawing (if different fro®, the originator) 
Multiple drawing reviews by persons not directly concerned with either the production 
of the drawing, or the subsequent implementation of it during the hardware production 
phase of a program, is not required; and this practices is to be discouraged. 

2. 6. 7.1. R ationale : Fewer signatories for each drawing produced, facilitates the 

release process by saving handling time, and hence costs. 

2 ' 7, - ^ - g - win g Ch ahge Control : Drawing changes shall be kept to that minimum essential 
to ensure that an acceptable end product can be obtained from use of the drawing in 
question to secure that objective. 


2«7«1'« "R ed-lining " ; As changes become necessary, drawings shall be red-lined to 
permit rapid incorporation of the changes considered essential. When the change process 
is completed, a simple record shall be kept on the face of the drawing whei-e this is 
practical, recording the extent and date of the Red-Line change made. Where the size 
of the drawing, or the scope of the changes to be executed, or both render this 
face record impractical, a simple system of recording by use of typed, or pencil- 
lettered, Design Change Notices will be inaugurated. These DC Ns will be attached to 
the drawing which which they are associated, as part of the overall drawing package. 
When all changes have been incorporated on the drawing, a final fair copy will be 
prepared for final delivery to the customer if such is required contractually. 
Red-Lining, if used too liberally on a drawing, may tend to confuse the information 
contained thereon. Engineering judement is to be used, to limit the number of Red-Line 
changes inscribed. When the number of Red-Lined changes is such as to confuse the 
information, a simple Engineering Order (EO) will be issued requiring a new drawing, 
which incorporates all changes to date. 
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2. 7. 1.1. Rationale ; The method of expeditious handling of drawing changes, while 
not directly a part of the processes by which drawings are produced, is a major 
factor influencing drawing update, release, and availability. Red-Lining, as a pract- 
ice is well established, and facilitates rapid handling of changes, particularly 
for in-process manufactuirng drawings. Historically, Red-Lining has not been pro- 
scribed by either Category, or Forms 2 and 3 of MIL-D-1000, but tacit disapproval of 
the practice has been the accepted norm. NASA will specify in the RFP that Red-Lining 
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IS DISALLOWED FOR A GIVEN PROGRAM/CONTRACT, SHOULD THIS DECISION BE TAKEN FOR ANY 
REASON BY THE NASA CENTER INVOLVED . In all cases where no such dis-allowance is 
specified, the practice is to be undertaken in accordance with the requirements of 
this specification. 

2.8. New Symbology : In all cases where new symbology is developed, and required to 
be used for a drawing, or drawings, a detailed list of the new symbols contemplated 
for use, will be prepared and submitted to NASA for approval. Provisional approval 
to permit immediate, use will be furnished within not more than 3 working days, and 
formal approval within 30 days of submittal of such a list. 

2.8.1. Rationale : Symbology is dynamic in nature, as the state-of-the-art advances. 

Changes are particularly rapid in the area of electronic symbology, as LSI, MSI, 
and hybrid techniques, become increasingly prevalent. Where new symbols are evolved, 
listed, and submitted to NASA for approval, NASA shall undertake the task of up-dating 
those documents listing current symbology, and imposing these on contracts. 

2.9. Quality Assurance Provisions : Prior to the release of each drawing package. 

Quality Assurnace Inspectors/personnel shall inspect the package to determine that: 

o The correct number of drawings comprising the package is present 
o All changes have been duly incorporated, and that all drawings are 
correctly numbered and titled per the Engineering Drawing List (EDL) 
o No damage, defacement, or degradation of content has taken place, 
o That the package is in all respects ready for use/delivery to the 
customer. 
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A suitable inspection or QA stamp shall be affixed to the package, denoting that 
QA has approved the package for use/delivery. A Quality Assurance person shall 
sign the conveyance transferring ownership of the drawing package to the customer 

(NASA), i.e. Form D.D.2^0 or other, in the presence of the Government representative 
accepting the drawing package. 

3*0 Preparation for Delivery : 

3.1. Microfilm : Drawings which have been subjected to a microfilming reduction 

process, by contractual requirement, shall be packaged in a manner based on 

the provisions of MIL-M-9368; which document shall be used as a reference and guide. 

3.2. Reproducible, and Non-Reproducible Drawings, Including Originals : 

li/hen delivery of such drawings is a contractual requirement, they shall be packaged 
for shipment and delivery in a manner based upon the methods set forth in MIL-D-£1*80. 
This document shall be used as a reference and guide for the preparation process. 
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